I propose two practical advances to the argument-based approach to validity: developing a living document and incorporating preregistration. First, I present a potential structure for the living document that includes an up-to-date summary of the validity argument. As the validation process may span across multiple studies, the living document…

The method of web probing integrates cognitive interviewing techniques into web surveys and is increasingly used to evaluate survey questions. In a usual web probing scenario, probes are administered immediately after the question to be tested (concurrent probing), typically as open-ended questions. A second possibility of administering probes is…

The use of social media platforms as a means of data collection for researchers is increasing. With the number of active users of Facebook across the globe exceeding two billion, the platform offers a means to reach and hear from diverse populations and gather information-rich data. Despite an increase of researchers utilising Facebook and…

Written with faculty in mind, "Assessment by Design" is a practical resource that will also be useful to student affairs staff and administrators dedicated to using assessment to improve learning in curricular and co-curricular settings. This book presents the Cycle of Assessment as a framework that supports assessment in service of…

This research brief describes recent methodological initiatives with the Public Libraries Survey. It describes how starting with the 2022 data, Institute of Museum and Library Services (IMLS) updated geographic identifiers to better align Census Bureau geography types with the library's legal service area, with the goal of enabling data users to…

The article answers the research question: under what conditions and with what methods would museum visitors feel comfortable (and not comfortable) sharing sensitive information for the purposes of museum research or evaluation? We ground our study in literature about sharing personal information in person, online, and in the context of digital…

Recent years have witnessed an increasing emphasis on integrating computational thinking into school curriculums. This study deals with suggesting a course model including data collection tools for evaluating teachers' pedagogical content knowledge in teaching computational thinking via teaching robot programming. Taking the advantages of virtual…

The world has experienced rapid changes, leading to pressing issues such as environmental degradation, social inequality, and resource depletion. As a transdisciplinary field, evaluation has emerged as a crucial tool in addressing these challenges and promoting systemic change. However, concerns have been raised regarding the field's capacity to…

Background: Speech-language pathologists often multitask in order to be efficient with their commonly large caseloads. In stuttering assessment, multitasking often involves collecting multiple measures simultaneously. Aims: The present study sought to determine reliability when collecting multiple measures simultaneously versus individually.…

Even in "routine" years, the principal's attention to climate matters for wellness, and it matters for outcomes. School climate is a critical factor supporting the quality of student learning, particularly in times of stress. As a result, districts across the nation have dedicated considerable resources to tools used to assess climate,…

Making evidence-based decisions in schools seems like an impossible task when hundreds of activities are available, the evidence base fails to address most of them, and budget deadlines are immovable. In addition, different stakeholders hold different views on what it means for a program, strategy, or intervention to "work." Fiona…

Big international assessment organisations like Cambridge Assessment have long held considerable amounts of data. When text is produced digitally, we can do more with it. Surprisingly to many, there have been examples of automatic scoring of extended writing for around 20 years, though what works well in one context may not be applicable in all…

The book provides a grounded, narrative exploration of contemporary qualitative PhD research in the fields of language education and applied linguistics. The chapters are authored by current and former PhD candidates studying in New Zealand, with commentaries from international experts in the field. The book contains ten chapters in addition to…

This case study explains how to gather web metrics to measure the size, investment, and popularity of K-12 school system and higher education websites. This case study will first explain and define web metrics in detail, such as keywords, traffic, search engine optimization, and other terminology crucial for educational researchers to understand…

Partnership models have been effective across many areas of higher education such as involving students as teaching and learning consultants, in course design and redesign, and as co-instructors. However, there are few systems-level (i.e., entire programs or institutions) examples of partnership work and virtually none in systems-level assessment.…