The multistage testing (MST) design has been gaining attention and popularity in educational assessments. For testing programs that have small test-taker samples, it is challenging to calibrate new items to replenish the item pool. In the current research, we used the item pools from an operational MST program to illustrate how research studies…

Response styles introduce construct-irrelevant variance as a result of respondents systematically responding to Likert-type items regardless of content. Methods to account for response styles through data analysis as well as approaches to mitigating the effects of response styles during data collection have been well-documented. Recent approaches…

Student evaluations of teaching (SETs) are a ubiquitous feature of higher education. However, scholars have presented numerous challenges to the accuracy, validity, reliability and objectivity of SETs as a measure of teaching effectiveness. Given the potential for bias, the use of SETs in professional review may constitute a form of institutional…

This report includes information on the design and methodology of the 2021-2022 Study of Family and Staff Well-Being in Head Start Family and Child Experiences Survey Programs (the 2021-2022 Study) and presents key findings from the study's spring 2022 data collection with Head Start program directors, center directors, and teachers. The 2021-2022…

Purpose: This article presents focus groups as a method to enhance questionnaire design to frame and test items for a mass survey on corruption and austerity in democracy. Such methodology lacks systematization when it comes to the description of implementation procedures and the discussion of effective contribution to the development of survey…

K-12 Engineering Education has placed a lot of attention on students' attitudes or predispositions towards science and technology. However, most assessment methods are focused on STEM as a whole or only on technology. In this article, we will discuss the instrument called Technology and Engineering Attitude Scale (TEAS) which focuses on attitudes…

The aim of the present study was to investigate the properties of paper-and-pencil data collection instruments developed to measure Computational Thinking (CT) based on several variables. Thus, keywords were identified and used in searches conducted in various databases. The outcomes of the search were analyzed based on the inclusion/exclusion…

Response process validity evidence provides a window into a respondent's cognitive processing. The purpose of this study is to describe a new data collection tool called a whole-class think aloud (WCTA). This work is performed as part of test development for a series of problem-solving measures to be used in elementary and middle grades. Data from…

Early childhood educators play essential roles in providing stable and high-quality child care for young children and supporting their development and growth. As is also true nationwide, historically low wages in the District of Columbia have led to challenges in compensating and retaining qualified early childhood educators, and in turn, building…

The "Morbidity and Mortality Weekly Report" ("MMWR") series of publications is published by the Office of Science, Centers for Disease Control and Prevention (CDC), U.S. Department of Health and Human Services. Articles included in this supplement are: (1) Overview and Methods for the Youth Risk Behavior Surveillance System --…

Digital literacy is essential for scientific literacy in a digital world. Although the NGSS Practices include many activities that require digital literacy, most studies have examined digital literacy from a generic perspective rather than a curricular context. This study aimed to develop a self-report tool to measure elements of digital literacy…

Mixed methods-grounded theory (MM-GT) has emerged as a promising methodology that intersects the value of mixed methods with rigorous qualitative design. However, recent reviews have found that MM-GT empirical studies tend to lack procedural details. The purpose of this article is to apply the "best practices" for conducting MM-GT in a…

Short message service (SMS or text messaging) surveys can collect data quickly and inexpensively. However, SMS surveys have space constraints that pose difficulties to questionnaire designers: Questions can only be 160 characters or less, surveys must be short, and many respondents use phones with small screens. In this article, we compare SMS and…

This study investigated the performance of four widely used data-collection designs in detecting test-mode effects (i.e., computer-based versus paper-based testing). The experimental conditions included four data-collection designs, two test-administration modes, and the availability of an anchor assessment. The test-level and item-level results…

This study investigates the effects of a scenario-based assessment design on students' writing processes. An experimental data set consisting of four design conditions was used in which the number of scenarios (one or two) and the placement of the essay task with respect to the lead-in tasks (first vs. last) were varied. Students' writing…