The Energy and Momentum Conceptual Survey (EMCS) is a multiple-choice survey that contains conceptual problems involving a variety of energy and momentum concepts covered in a typical introductory physics course for science and engineering majors. Prior studies suggest that many concepts on the survey are challenging for introductory physics…

In the present study, we recorded the eye movements of 20 criterion-based selected trainee teachers in economics while they responded to 25 single choice (SC) items in an economic content knowledge (CK) test and rated their confidence for each response in a digital assessment. By using a multilevel modeling approach with crossed random effects, we…

Disengaged responding poses a severe threat to the validity of educational large-scale assessments, because item responses from unmotivated test-takers do not reflect their actual ability. Existing identification approaches rely primarily on item response times, which bears the risk of misclassifying fast engaged or slow disengaged responses.…

Providing students with feedback on their performance is a critical part of enhancing student learning in chemistry and is often integrated into homework assignments, quizzes, and exams. However, not all feedback is created equal, and the type of feedback the student receives can dramatically alter the utility of the feedback to reinforce correct…

Research has suggested that changing one's answer on multiple-choice examinations is more likely to lead to positive academic outcomes. This study aimed to further understand the relationship between changing answer selections and item attributes, student performance, and time within a population of 158 first-year medical students enrolled in a…

Students' digital technology literacy is required to develop technology-based teaching skills as prospective teachers. Therefore, students' digital technology literacy profiles must be known as a basis for learning policies. This research aims to describe the digital technology literacy profile of students as prospective elementary school…

Multidimensional forced-choice (MFC) items have been found to be useful to reduce response biases in personality assessments. However, conventional scoring methods for the MFC items result in ipsative data, hindering the wider applications of the MFC format. In the last decade, a number of item response theory (IRT) models have been developed,…

Most biology undergraduates learn about mutations in multiple classrooms throughout their college career. Understanding personalised genome test results, genome editing controversies, and the appearance of new variants of viruses or antibiotic resistant bacteria all require foundational knowledge about mutations. However, the abstract nature of…

The purpose of this study was to compare the quality of multiple choice questions (MCQs) developed using automated item generation (AIG) versus traditional methods, as judged by a panel of experts. The quality of MCQs developed using two methods (i.e., AIG or traditional) was evaluated by a panel of content experts in a blinded study. Participants…

This document provides advice on moving from in-place, closed-book examinations to open-book, remote assessment. It offers conceptual considerations, practical tips, and discusses the relative merits of two common approaches to open-book exams: those with strict time limits, and those with broad time limits ('take-home' exams).

This work proposes a new approach for measuring long-term conceptual knowledge based on the after-instruction evolution of students' answers to a research-based, multiple-choice, single-response test. The method allows for a quantitative determination of the fraction of students that, after instruction, attain long-lasting and temporary learnings,…

This paper deals with the advantages and disadvantages to both the faculty member and to the students of using the answer-until-correct multiple-choice test format for a class specifically designed for non-STEM majors, who need a breath of knowledge science course. Comments from students are given about why they prefer answer-until-correct…

Association questions (AQs) are a novel form of multiple-choice questions (MCQs). To answer an AQ, learners must recall the concepts denoted by the given terms, affirm their connections, and then select the term with a denotation that is "less connected" with other concepts. This research hypothesizes that the use of AQ tests for…

The existence of outliers in the anchor items can be detrimental to the estimation of examinee ability and undermine the validity of score interpretation across forms. However, in practice, anchor item performance can become distorted due to various reasons. This study compares the performance of modified "INFIT" and "OUTFIT"…

This article investigates the concern that assessment of metacognitive resolution (or relative accuracy--often evaluated by gamma correlations or signal detection theoretic measures such as d[subscript a]) is vulnerable to an artifact due to guessing that differentially impacts low as compared to high performers on tasks that involve…