We address two persistent challenges in large-scale assessments of the Next Generation Science Standards: (a) the validity of score interpretations that target the standards broadly and (b) how to structure claims for assessments of this complex domain. The NGSS pose a particular challenge for specifying claims about students that evidence from…

Background: Artificial intelligence (AI) is rapidly changing communication and technology-driven content creation and is also being used more frequently in education. Despite these advancements, AI-powered automated scoring in international large-scale assessments (ILSAs) remains largely unexplored due to the scoring challenges associated with…

Research-validated multiple-choice questions comprise an easy-to-implement instructional tool that serves to scaffold student learning and formatively assess students' knowledge. We present findings from the implementation, in consecutive years, of a research-validated multiple-choice question sequence [referred to in this study as a clicker…

International Large-Scale Assessments (ILSAs) have had significant impact on education policy across the globe. But has interest in ILSAs now started to wane? This paper presents new evidence on this issue, exploring how the amount of attention three major ILSAs receive compares across countries, between studies and over time. Using information on…

This paper presents improvements made to a computerized adaptive testing (CAT)-based version of the FCI (FCI-CAT) in regards to test security and test efficiency. First, we will discuss measures to enhance test security by controlling for item overexposure, decreasing the risk that respondents may (i) memorize the content of a pretest for use on…

Digital technologies have the potential to revolutionize education by enhancing quality, fairness, and efficiency. However, equitable access to these technologies remains a challenge. ILSAs (international large-scale assessments) have shown that the relationship between digital use and performance varies across countries and over time. To fully…

Research-based multiple-choice questions implemented in class with peer instruction have been shown to be an effective tool for improving students' engagement and learning outcomes. Moreover, multiple-choice questions that are carefully sequenced to build on each other can be particularly helpful for students to develop a systematic understanding…

Many studies have examined the structure and properties of the Force Concept Inventory (FCI); however, far less research has investigated the Force and Motion Conceptual Evaluation (FMCE). This study applied Multidimensional Item Response Theory (MIRT) to a sample of N = 4528 FMCE post-test responses. Exploratory factor analysis showed that 5, 9,…

Collaborative problem solving (CPS) is an important 21st-century skill that is crucial for both career and academic success. However, developing a large-scale and standardized assessment of CPS that can be administered on a regular basis is very challenging. In this report, we introduce a set of psychometric considerations and a general scoring…

Giftedness in science today is largely measured by various kinds of standardized tests--IQ tests, SATs, ACTs, GREs, and so forth. For example, many STEM (science, technology, engineering, math) gifted programs rely at least in part on IQ tests or the SAT for identifying students as gifted. It might be useful to supplement such standard measures…

This article presents the author's response to the article "Taking Stock: Existing Resources for Assessing a New Vision of Science Learning" by Alonzo and Ke (this issue), which identifies numerous challenges that the Next Generation Science Standards (NGSS) pose for large-scale assessment. Jill Werthem comments that among those…

The instructional lab setting has been found to be dominated by prescribed tasks and pre-prepared lab kits. This was explained by teachers' need to guide students to simultaneously progress through a lab curriculum, which prompts them to standardize the lab experience. Nevertheless, prominent professional associations have persistently called to…

Concept inventories (CIs) are commonly used in pre-post instruction to study student conceptual change. For consistency in assessment interpretation, a CI's assessment construct is desired to maintain invariance across different test times. In this study, the longitudinal measurement invariance (LMI) analysis under the confirmatory factor analysis…

Low arousal states (especially boredom) have been shown to be more deleterious to learning than high arousal states, though the latter have received much more attention (e.g., test anxiety, confusion, and frustration). Aiming at profiling arousal in the classroom (how active students are) and examining how activation levels relate to achievement,…

The International Large-Scale Assessment (ILSA) is a rapidly growing field in education which has gained considerable attention of stakeholders across the globe. Historically, ILSA emerged from the developed context which has drifted to the developing contexts in a short span of time due to globalisation. Pakistan has participated in ILSA (i.e.,…