This is the second article in a series of three in this special issue on establishing a boundary object to foster network health and development. The first article laid out the theoretical rationale for an Improvement Network Health and Development Framework. This article details the efforts to develop a set of practical measures tied to this…

When comparing relations and means of latent variables, it is important to establish measurement invariance (MI). Most methods to assess MI are based on confirmatory factor analysis (CFA). Recently, new methods have been developed based on exploratory factor analysis (EFA); most notably, as extensions of multi-group EFA, researchers introduced…

This article builds upon a proiminent definition of construct validity that focuses on variation in attributes causing variation in measurement outcomes. This article synthesizes the defintion and uses Rasch measurement modeling to explicate a modified conceptualization of construct validity for assessments of developmental attributes. If…

On page 1 of his classic text, Millsap (2011) states, "Measurement invariance is built on the notion that a measuring device should function the same way across varied conditions, so long as those varied conditions are "irrelevant" [emphasis added] to the attribute being measured." By construction, measurement invariance…

The Political Identity and Relational Impact Scale (PIRI) is a self-report scale of personal and relational responses to political conflict. This article describes the development and exploratory factor analysis of the scale. The selected model consists of 30 items and a four-factor structure with high internal consistency.

Evidence-based assessment (EBA) requires that investigators employ scientific theories and research findings to guide decisions about what domains to measure, how and when to measure them, and how to make decisions and interpret results. To implement EBA, investigators need high-quality assessment tools along with evidence-based processes. We…

Self-report is required to assess mental states in nuanced ways. By implication, self-report is indispensable to capture the psychological processes driving human learning, such as learners' emotions, motivation, strategy use, and metacognition. As shown in the contributions to this special issue, self-report related to learning shows convergent…

Eye tracking has become an increasingly popular research methodology among language researchers to examine online cognitive processing of second-language (L2) speakers and bilinguals. As the scope of eye-tracking research expands, there is a need to ensure that the methodology is used properly, so as to safeguard the validity of research findings…

A great deal has been written on the topic of test validity. Guiding our work at ACT are "The Standards for Educational and Psychological Testing" (2014), which outlines best practices in test development and validation. As ACT transitions from an assessment company to a learning, measurement, and navigation organization, a framework for…

Dynamic Measurement Modeling (DMM) is a psychometric paradigm that uses longitudinal data to estimate students' capacity to learn over the course of an educational program (i.e., growth scores). Here, we provide justification for this approach in health professions education and demonstrate its proof of concept with three time-points of USMLE Step…

A multistage adaptive testing (MST) design was implemented for the Programme for the International Assessment of Adult Competencies (PIAAC) starting in 2012 for about 40 countries and has been implemented for the 2018 cycle of the Programme for International Student Assessment (PISA) for more than 80 countries. Using examples from PISA and PIAAC,…

Ultimately, measures in research, testing, assessment and evaluation are used, or have implications, for ranking, intervention, feedback, decision-making or policy purposes. Explicit recognition of this fact brings the often-ignored and sometimes maligned concept of consequences to the fore. Given that measures have personal and social…

In 2017, the Office of Planning, Research, and Evaluation (OPRE) in the Administration for Children and Families (ACF) funded Mathematica to conduct the Infant and Toddler Teacher and Caregiver Competencies (ITTCC) project. The project aims to examine existing efforts across states, institutions of higher education, professional organizations, and…

While basic proficiency in mathematics, reading, and writing is essential, educators and parents alike would more likely list characteristics like perseverance, self-control, creativity, time management, leadership, conscientiousness, and being an effective collaborator when considering what is most important for success in school, work, and life.…

Like instruments used in afterschool programs to assess children's social and emotional growth or to evaluate staff members' performance, instruments used to evaluate program quality should be free from bias. Practitioners and researchers alike want to know that assessment instruments, whatever their type or intent, treat all people fairly and do…