This is the second of five papers detailing our national study of general education astronomy students' conceptual and reasoning difficulties with cosmology. This article begins our quantitative investigation of the data. We describe how we scored students' responses to four conceptual cosmology surveys, and we present evidence for the inter-rater…

This study compares the psychometric utility of Classical Test Theory (CTT) and Item Response Theory (IRT) for scale construction with data from higher education student surveys. Using 2008 Your First College Year (YFCY) survey data from the Cooperative Institutional Research Program at the Higher Education Research Institute at UCLA, two scales…

In much of the world, the issue of accountability and measurement of educational outcomes is highly controversial. Exit testing is part of the movement to ascertain what students have learned and hold institutions and teachers to account. However, compared to the large number of teacher washback studies, learner washback research is lacking…

Any meaningful initiative to improve the teaching and learning in introductory control systems courses needs a clear test of student conceptual understanding to determine the effectiveness of proposed methods and activities. The authors propose a control systems concept inventory. Development of the inventory was collaborative and iterative. The…

Three test construction strategies are described and illustrated in the development of the Verb Interest Test (VIT), an inventory that assesses vocational interests using verbs. Verbs might be a promising alternative to the descriptions of occupational activities used in most vocational interest inventories because they are context-independent,…

The validity of cognitive ability tests is often interpreted solely as a function of the cognitive abilities that these tests are supposed to measure, but other factors may be at play. The effects of test anxiety on the criterion related validity (CRV) of tests was the topic of a recent study by Reeve, Heggestad, and Lievens (2009) (Reeve, C. L.,…

This chapter introduces theory that undergirds the role of testing adaptations in assessment, provides examples of item modifications and testing accommodations, reviews research relevant to each, and introduces a new paradigm that incorporates opportunity to learn (OTL), academic enablers, testing adaptations, and inferences that can be made from…

Equating is an essential tool in educational assessment due the critical role it plays in several key areas: establishing validity across forms and years; fairness; test security; and, increasingly, continuity in programs that release items or require ongoing development. Although the practice of equating is rooted in long standing practices that…

Two different traditions of response-time (RT) modeling are reviewed: the tradition of distinct models for RTs and responses, and the tradition of model integration in which RTs are incorporated in response models or the other way around. Several conceptual issues underlying both traditions are made explicit and analyzed for their consequences. We…

In this technical report, we describe the results of a study of mathematics items written to align with the Common Core State Standards (CCSS) in grades 6-8. In each grade, CCSS items were organized into forms, and the reliability of these forms was evaluated along with an experimental form including items aligned with the National Council of…

It is a bitter reality that the curricula and traditional pedagogy prevailing in public schools of Pakistan in general and Sindh in particular do not incorporate the algebraic concepts properly. Both the content and the presentation therein cannot be considered up to the mark, thereby making "Algebra" a tough and dry subject. This…

Standardizing aspects of assessments has long been recognized as a tactic to help make evaluations of examinees fair. It reduces variation in irrelevant aspects of testing procedures that could advantage some examinees and disadvantage others. However, recent attention to making assessment accessible to a more diverse population of students…

In information-rich environments, schools are often presented with a myriad of data from which decisions need to be made. The use of the information on a classroom level may be facilitated if performance could be described in terms of levels of proficiency or benchmarks. The aim of this article is to explore benchmarks using data from a monitoring…

Purpose: The main aim of this paper is to develop guidelines on the critical issues to consider in research design in an action research (AR) environment for SME network capability development. Design/methodology/approach: The issues in research design for AR studies are developed from the authors' experience in running learning sets but, in…

This study was inspired by the final regulations for the No Child Left Behind Act (NCLB) indicating that each state has the option to develop a new assessment for students whose disabilities have kept them from obtaining proficiency. Sets of high school science achievement items were enhanced for the new test. A 3-by-2, within subjects,…