The Cross-Classified Mixed Effects Model (CCMEM) has been demonstrated to be a flexible framework for evaluating reliability by measurement specialists. Reliability can be estimated based on the variance components of the test scores. Built upon their accomplishment, this study extends the CCMEM to be used for evaluating validity evidence.…

The purpose of this National Assessment of Educational Progress (NAEP) Achievement Levels Validity Argument Report is to synthesize evidence currently available to address the validity of the interpretations and uses of the NAEP Achievement Levels. Validity is the extent to which theory and evidence supports or refutes proposed and enacted test…

This study aimed at examining the suitability of questionnaire instruments commonly used in large-scale assessments for measuring non-cognitive school effectiveness criteria. It focused on questions of reliability and validity for capturing changes in students within schools across time and the instruments' sensitivity to school effects. The aim…

Recognizing the limitations of the existing frameworks and models applied to estimate the learning loss caused by COVID-19 across the world by different studies, and the urgent need for estimating such loss at the different education levels, this paper, employs the Education Value Chain Analysis ( EVCA) approach as a new paradigm, attempts to…

The Student Background survey administered along with achievement tests in studies of the International Association for the Evaluation of Educational Achievement includes scales of student motivation, competence, and attitudes toward mathematics and science. The scales consist of positively- and negatively keyed items. The current research…

This paper describes a statistical linking between the 2011 National Assessment of Educational Progress (NAEP) in Grade 4 reading and the 2011 Progress in International Reading Literacy Study (PIRLS) in Grade 4 reading. The primary purpose of the linking study is to obtain a statistical comparison between NAEP (a national assessment) and PIRLS (an…

This study presents a conceptual and evidentiary study pertaining to the micro and macro level factors influencing the achievement gap among migrant children from a cross-national standpoint. First, this study provides the most systematic investigation of the immigrant educational paradox theory, which asserts that on average, immigrants'…

International large-scale assessments of achievement often have a large degree of differential item functioning (DIF) between countries, which can threaten score equivalence and reduce the validity of inferences based on comparisons of group performances. It is important to understand potential sources of DIF to improve the validity of future…

Although International Association for the Evaluation of Educational Achievement-pioneered international large-scale assessment (ILSA) of education is now a well-established science, non-practitioners and many users often substantially misunderstand how large-scale assessments are conducted, what questions and challenges they are designed to…

In this article I advocate for a new discussion in the field of international large-scale assessments; one that calls for a reexamination of international large-scale assessments (ILSAs) and their use. Expanding on the high-quality work in this special issue I focus on three inherent limitations to international large-scale assessments noted by…

Research indicates the need to further examine the dimensions of reading motivation. A clear theoretical basis is necessary for conceptualizing reading motivation and considering contextual differences therein. The present study develops and validates the SRQ-Reading Motivation, a questionnaire measuring recreational and academic reading…

The judgmental approaches used by the Governing Board to set National Assessment of Educational Progress achievement levels have been the subject of a great deal of controversy and have sometimes led to results that were considered "unreasonable." The NCES requirement that the achievement levels be used on a trial basis and be…

The ranking of the United States in major international tests such as the Progress in International Reading Literacy Study (PIRLS), Trends in International Mathematics and Science Study (TIMSS), and Program for International Student Assessment (PISA) is used as the driving force and rationale for the current educational reforms in the United…

The purpose of the present study was to examine validity aspects of teachers' judgements of pupils' reading skills. Data come from Sweden's participation in the Progress in International Reading Literacy Study (PIRLS) 2001, for Grades 3 and 4. For pupils at the same achievement levels, as measured by PIRLS 2001 test, teachers' judgements of…

In international achievement studies, questionnaires typically ask about the presence of particular household assets in students' homes. Responses to the assets questions are used to compute a total score, which is intended to represent household wealth in models of test performance. This study uses item analysis and confirmatory factor analysis…