Despite their drawback, multiple-choice questions are an enduring feature in instruction because they can be answered more rapidly than open response questions and they are easily scored. However, it can be difficult to generate good incorrect choices (called "distractors"). We designed an algorithm to generate distractors from a…

Conceptual change is assumed to be a potential teaching pedagogy to prepare the skills for anticipating the 21st century. A quasi-experimental method with pre-and post-test control group design was applied to examine the effectiveness of integrating the 3-2-1 reading technique with refutation text on students' conceptual change of buoyancy. The…

This paper extends prior work establishing an operationalized framework of mathematical sense making (MSM) in physics. The framework differentiates between the object being understood (either physical or mathematical) and various tools (physical or mathematical) used to mediate the sense-making process. This results in four modes of MSM that can…

The Conceptual Survey of Electricity and Magnetism (CSEM) is a widely used multiple-choice instrument measuring a student's conceptual understanding of electricity and magnetism. This study applied modified module analysis (MMA) and modified module analysis-partial (MMA-P), network analytic methods that identify groups of correlated responses, to…

Black women have been making successful strides in mathematics for decades; however, they continue to be underrepresented in mathematics and other STEM fields. According to Young et al. (2017), Black girls and women perform lower in mathematics than all other racial gender groups except for Black males. Considering the stakes for Black girls and…

Grading is often a faculty member's least favorite chore, especially in engineering where open-ended problems prevail. For this reason, multiple-choice test items could be a popular alternative for assessing learning and understanding. In addition, most Learning Management Systems allow the instructor to create multiple-choice questions to be…

This study expands arguments calling for a more rigorous approach to high-frequency vocabulary list-based learning in EFL learning environments. Test and flashcard item designs were validated through quantitative (midterm and final test) and qualitative (survey) results to explore the impact of digital flashcards designed to build recall-level…

Purpose: This study attempts to evaluate the effect of the corpus-based inductive teaching approach with multiple academic corpora (PICA, CAEC and Oxford Corpus of Academic English) and conventional deductive teaching approach (i.e., multiple-choice items, filling the gap, matching and underlining) on learning academic collocations by Iranian…

Language instruction at the Defense Language Institute Foreign Language Center (DLIFLC), as it does at colleges and universities, focuses on developing linguistic competence. There is no textbook or teaching materials that explicitly and systematically teach students the pragmatic knowledge of the target language. Such knowledge is either…

This research aimed to investigate animation-based achievement tests according to the item format, psychometric features, students' performance, and gender. The study sample consisted of 52 fifth-grade students in Samsun/Turkey in 2017-2018. Measures of the research were open-ended (OE), animation-based open-ended (AOE), multiple-choice (MC), and…

It is widely recognised that the development of higher order thinking skills is a fundamental goal of higher education. There are a variety of online tools that assist the development of student higher order thinking. In this paper, a process of scaffolding for the writing of higher-order questions enabled through peer learning activities is…

Even though the impact of the position of response options on answers to multiple-choice items has been investigated for decades, it remains debated. Research on this topic is inconclusive, perhaps because too few studies have obtained experimental data from large-sized samples in a real-world context and have manipulated the position of both…

This study describes the characteristics of the test and its items used in the national-standardized school examination by applying classical test theory and focusing on the item difficulty, item discrimination, test reliability, and distractor analysis. We analyzed response data of 191 12th graders from one of public senior high schools in…

Data science and informatics tools are developing at a blistering rate, but their users often lack the educational background or resources to efficiently apply the methods to their research. Training resources and vignettes that accompany these tools often deprecate because their maintenance is not prioritized by funding, giving teams little time…

In this quasi-experimental study, the effects of lexical coverage through pseudo word manipulation in dialogue comprehension are investigated. Forty-four first-year students in a Japanese university listened to five dialogues at different lexical coverage levels: 98%, 95%, 90%, 85%, and 83%. The results of the comprehension tests confirm the…