Multilevel modeling (MLM) is frequently used to detect cluster-level group differences in cluster randomized trial and observational studies. Group differences on the outcomes (posttest scores) are detected by controlling for the covariate (pretest scores) as a proxy variable for unobserved factors that predict future attributes. The pretest and…

Cognitive diagnostic models (CDM) have been around for more than a decade but their application is far from widespread for mainly two reasons: (1) CDMs are novel, as compared to traditional IRT models. Consequently, many researchers lack familiarity with them and their properties, and (2) Software programs doing CDMs have been expensive and not…

This article illustrates how a new framework for conceptualising comparability has the potential to help assessment professionals to understand and to conduct debate on linking theory and practice. The framework was used as a lens through which to study a corpus of research reports, from which a narrative was constructed to characterise the…

We developed and evaluated an Oceanography Concept Inventory (OCI), which used a mixed-methods approach to test student achievement of 11 learning goals for an introductory-level oceanography course. The OCI was designed with expert input, grounded in research on student (mis)conceptions, written with minimal jargon, tested on 464 students, and…

This study aimed to compare the accuracy of the test scores as results of Test of English Proficiency (TOEP) based on paper and pencil test (PPT) versus computer-based test (CBT). Using the participants' responses to the PPT documented from 2008-2010 and data of CBT TOEP documented in 2013-2014 on the sets of 1A, 2A, and 3A for the Listening and…

Although literacy achievement has improved in Chile, adolescents' underperformance in reading comprehension is still a serious concern. In English, core academic-language skills (CALS) have been found to significantly predict reading comprehension, even controlling for academic vocabulary knowledge. CALS are high-utility language skills that…

A classic topic in the fields of psychometrics and measurement has been the impact of the number of scale categories on test score reliability. This study builds on previous research by further articulating the relationship between item response theory (IRT) and classical test theory (CTT). Equations are presented for comparing the reliability and…

This paper explores the relation between problem solving learning environments (PSLEs) and assessment concepts. The general framework of evidence-centered assessment design is used to describe PSLEs in terms of assessment concepts, and to identify similarities between the process of assessment design and of PSLE design. We use a recently developed…

We report results of an exploratory validation study of the "Student Risk Screening Scale-Internalizing and Externalizing" (SRSS-IE) applied with the first sample of middle and high school students from nine middle and three high schools from three states. The "Student Risk Screening Scale" (SRSS) was modified to broaden the…

The purpose of the researcher in this quantitative study was to examine the relationship between principals' leadership practices, school culture, and student achievement as perceived by elementary school teachers. The researcher established the 5 research questions to: (a) determine the differences between high- and low-achievement schools on the…

In adolescence research, the treatment of measurement reliability is often fragmented, and it is not always clear how different reliability coefficients are related. We show that generalizability theory (G-theory) is a comprehensive framework of measurement reliability, encompassing all other reliability methods (e.g., Pearson "r,"…

Background/Context: This article explains the idea of a neopragmatic postmodernist test theory and offers some thoughts about what changing notions concerning the nature of and meanings assigned to knowledge imply for educational assessment, present and future. Purpose: Advances in the learning sciences--particularly situative and sociocognitive…

Learning standards across the United States have increasingly called for history students to engage in aspects of "historical thinking," a term used to describe the complex disciplinary processes that historians use to make sense of the past. Although students are expected to learn these complex processes, little is known about how to…

Cognitive diagnosis models (CDMs) continue to generate interest among researchers and practitioners because they can provide diagnostic information relevant to classroom instruction and student learning. However, its modeling component has outpaced its complementary component-test construction. Thus, most applications of cognitive diagnosis…

This study assesses the Japanese translation of the Force and Motion Conceptual Evaluation (FMCE). Researchers are often interested in comparing the conceptual ideas of students with different cultural backgrounds. The FMCE has been useful in identifying the concepts of English-speaking students from different backgrounds. To identify effectively…