The Cross-Classified Mixed Effects Model (CCMEM) has been demonstrated to be a flexible framework for evaluating reliability by measurement specialists. Reliability can be estimated based on the variance components of the test scores. Built upon their accomplishment, this study extends the CCMEM to be used for evaluating validity evidence.…

In this paper, we investigated the impact of an online classroom-based reading assessment on implementing practices in reading instruction among 30 EFL learners in an intermediate reading course at a public university in East Java, Indonesia. Our study aimed to develop an online classroom-based reading assessment and evaluate its efficacy in…

Despite the increased use of integrated tasks in high-stakes academic writing assessment, research on rating criteria which reflect the unique construct of integrated summary writing skills is comparatively rare. Using a mixed-method approach of expert judgement, text analysis, and statistical analysis, this study examines writing features that…

Cognitive diagnostic models (CDMs) have received much interest within the field of language testing over the last decade due to their great potential to provide diagnostic feedback to all stakeholders and ultimately improve language teaching and learning. A large number of studies have demonstrated the application of CDMs on advanced large-scale…

Computerized game-based assessment (GBA) system for screening reading difficulties may provide substantial time and cost benefits over traditional paper-and-pencil assessment while providing means also to individually adapt learning content in educational games. To study the reliability and validity of a GBA system to identify struggling readers…

The purpose of this study was to investigate the relationship between scores on two common curriculum-based measures (CBMs), oral reading fluency (ORF) and maze, and scores on the previous statewide assessment of reading in Oregon (Oregon Assessment of Knowledge and Skills-Reading/Literature [OAKS-R]). Data obtained from three cohorts of…

Dynamic measurement modeling (DMM) has been shown to improve the consequential validity of longitudinal mathematics assessment in the Early Childhood Longitudinal Study-Kindergarten (ECLS-K) database. Here, the authors demonstrate the capability of DMM to similarly improve the consequential validity of ECLS-K reading assessment through the…

Curriculum-Based Measurement of Oral Reading (CBM-R) is often used to monitor student progress and guide educational decisions. Ordinary least squares regression (OLSR) is the most widely used method to estimate the slope, or rate of improvement (ROI), even though published research demonstrates OLSR's lack of validity and reliability, and…

In the United States, educational thought leaders have called for higher expectations (Gordon Commission, 2010), more rigorous college and career readiness standards for K-12 education (National Governors Association Center for Best Practices & Council of Chief State School Officers, 2010), and new constructs such as 21st century skills…

The PM Benchmark Reading Assessment Resource (PM Benchmarks) provides information about primary students' level of achievement in reading accurately, fluently, and in understanding unseen texts. As part of the assessment, students have to orally retell what they have read and orally answer comprehension questions. This oral use of language…

The purpose of the study is to have a relook at the ESL reading comprehension assessment system for Malaysian Year Five students. Traditionally, the ESL teachers have been assessing and reporting on their primary year's students by merely giving a composite grade with some vague remarks. This process has been used and is still being employed in…

Two hundred and two (n = 202) sixth-grade students in social studies were administered a weekly vocabulary-matching curriculum-based measure (CBM) for 35 weeks. Students were also administered the Scholastic Reading Inventory (SRI), along with the annual state high-stakes test in Communication Arts. CBM scores were analyzed with respect to…

This paper examines how different test methods may tap different aspects of second language knowledge. It employs multiple-choice (MC) and constructed response (CR) items which yield distinct or convergent information in the computer delivered testing of English in its presentation of this factor. In order to examine the effects of test method, a…

In this article I advocate for a new discussion in the field of international large-scale assessments; one that calls for a reexamination of international large-scale assessments (ILSAs) and their use. Expanding on the high-quality work in this special issue I focus on three inherent limitations to international large-scale assessments noted by…

For decades, standardized reading comprehension tests have consisted of a series of passages and associated multiple-choice questions. Although widely used in and out of the classroom, there continues to be considerable disagreement regarding how or whether such tests have net value in the service of advancing educational progress in reading. This…