Objective To develop an online discrete choice experiment (DCE) to elicit preferences for the format and content of tobacco treatment and determine the feasibility of the survey in face-to-face online interviews among college students. format content Participants A convenience sample of 28 college students. Methods A pilot online DCE survey with…

The objective of this paper is to comment on the think-aloud methods presented in the three papers included in this special issue. The commentary offered stems from the author's own psychological investigations of unobservable information processes and the conditions under which the most defensible claims can be advanced. The structure of this…

Reading and writing instruction involve teachers guiding students to take on the actions and thinking processes of readers and writers. Think-alouds and write-alouds are research-supported strategies that promote student growth and achievement. Although professional development strategies are outlined in the research literature on using…

There is a need for assessment items that assess complex constructs but can also be efficiently scored for evaluation of teacher education programs. In an effort to measure the construct of teacher attentiveness in an efficient and scalable manner, we are using exemplar responses elicited by constructed-response item prompts to develop…

Response process validity evidence provides a window into a respondent's cognitive processing. The purpose of this study is to describe a new data collection tool called a whole-class think aloud (WCTA). This work is performed as part of test development for a series of problem-solving measures to be used in elementary and middle grades. Data from…

Measures of teacher mathematical knowledge are notoriously difficult to develop (e.g., Orrill et al., 2015). This is in part because of the multidimensional nature of teacher knowledge. As part of two separate projects being undertaken by this research team, we have attempted to write assessments of teacher pedagogical content knowledge (PCK) in…

This study touches upon three major themes in the field of information literacy (IL): the assessment of IL, the association between IL knowledge and skills, and the dimensionality of the IL construct. Three quantitative measures were developed and tested with several samples of university students to assess knowledge and skills for core facets of…

The use of computer-based tests (CBTs), for both formative and summative purposes, has greatly increased over the past years. One major advantage of CBTs is the easy integration of multimedia. It is unclear, though, how to design such CBT environments with multimedia. The purpose of the current study was to examine whether guidelines for designing…

Although an accurate measure of vocabulary size is integral to understanding the proficiency of language learners, the validity of multiple-choice (M/C) vocabulary tests to determine this has been questioned due to users guessing correct answers which inflates scores. In this paper the nature of guessing and partial knowledge used when taking the…

We describe the use of think alouds to examine substantive processes involved in performance on a formative assessment of computational thinking (CT) designed to support self-regulated learning (SRL). Our task design model included three phases of work on a computational thinking problem: forethought, performance, and reflection. The cognitive…

Using a multiphase approach, the purpose of the present study was to develop a psychometrically sound questionnaire to measure protégés' perceptions of peer athlete mentoring functions. Phase 1 consisted of three stages: (a) item development, (b) assessment of content validity via think-aloud interviews with peer mentored athletes, and (c)…

In line with the widespread proliferation of digital technology in everyday life, many countries are now beginning to use computer-based exams (CBEs) in their post-primary education systems. To ensure that these CBEs are delivered in a manner that preserves their fairness, validity, utility and credibility, several factors pertaining to their…

Artificial intelligence (AI) technologies have the potential to reduce the workload for the second language (L2) teachers and test developers. We propose two AI distractor-generating methods for creating Chinese vocabulary items: semantic similarity and visual similarity. Semantic similarity refers to antonyms and synonyms, while visual similarity…

Background: The development of a set of questions is a central element of examination development, with the validity of an examination resting to a large extent on the quality of the questions that it comprises. This paper reports on the methods and findings of a project that explores how educational examination question writers engage in the…

In this study, we describe the methodology used to identify and validate a set of expert-defined fraction subtraction related attributes. These attributes are expected to be mastered by 6th grade students toward proficiency in fraction subtraction. This research argues and demonstrates that state standards guiding subject instruction plays an…