Balancing assessment of learning outcomes with the expectations of students is a perennial challenge in education. Difficult exams, in which many students perform poorly, exacerbate this problem and can inspire a wide variety of interventions, such as a grading curve. However, addressing poor performance can sometimes distort or inflate grades and…

This study aimed to reveal the differences in individuals' abilities, their standard errors, and the psychometric properties of the test according to the two methods of applying the test (electronic and paper). The descriptive approach was used to achieve the study's objectives. The study sample consisted of 74 male and female students at the…

This study presents the development and validation of a novel multiple-choice test designed to assess university students' conceptual understanding of electric field, force, and flux. The test of understanding of electric field, force, and flux was constructed based on the results of previous studies using a phenomenographic approach to classify…

The quality of multiple-choice questions (MCQs) as well as the student's solve behavior in MCQs are educational concerns. MCQs cover wide educational content and can be immediately and accurately scored. However, many studies have found some flawed items in this exam type, thereby possibly resulting in misleading insights into students'…

This article describes the steps we went through in designing and validating an item bank to diagnose linguistic problems in the English academic writing of university students in Hong Kong. Test items adopt traditional item formats (e.g., MCQ, grammatical judgment tasks, and error correction) but are based on authentic language materials…

Multiple test tasks are available for assessing L2 collocation knowledge. However, few studies have investigated the characteristics of a variety of recognition and recall tasks of collocation simultaneously, and most research on L2 collocations has focused on verb-noun and adjective-noun collocations. This study investigates (1) the relative…

This article reports on a study which aimed to determine the effect of strength of accent on listening comprehension of interactive lectures. Test takers (N = 21,726) listened to an interactive lecture given by one of nine speakers and responded to six comprehension items. The test taker responses were analyzed with the Rasch computer program…

Cloze tests have been the subject of numerous studies regarding their function and use in both first language and second language contexts (e.g., Jonz & Oller, 1994; Watanabe & Koyama, 2008). From a validity standpoint, one area of investigation has been the extent to which cloze tests measure reading ability beyond the sentence level.…

We used a set of action verbs based on Bloom's taxonomy to assess learning outcomes in two college-level introductory psychology courses. The action verbs represented an acronym, IDEA, comprising skills relating to identifying, defining or describing, evaluating or explaining, and applying psychological knowledge. Exam performance demonstrated…

This study investigates university students' understanding of graphs in three different domains: mathematics, physics (kinematics), and contexts other than physics. Eight sets of parallel mathematics, physics, and other context questions about graphs were developed. A test consisting of these eight sets of questions (24 questions in all) was…

In this article we discuss the findings of our research on students' understanding of vector concepts in problems without physical context. First, we develop a complete taxonomy of the most frequent errors made by university students when learning vector concepts. This study is based on the results of several test administrations of open-ended…

Item Response Theory (IRT) models were applied to investigate the psychometric properties of the Arthur and Day's Advanced Progressive Matrices-Short Form (APM-SF; 1994) [Arthur and Day (1994). "Development of a short form for the Raven Advanced Progressive Matrices test." "Educational and Psychological Measurement, 54," 395-403] in order to test…

Research on study-time allocation has largely focused on agenda-based regulation, such as whether learners select items for study that are in their region of proximal learning. In 4 experiments, the authors evaluated the contribution of habitual responding to study-time allocation (e.g., reading from left to right). In Experiments 1 and 2,…

What do learners do when they control whether to engage in massed or spaced practice? According to theories by Son (2004) and by Metcalfe and Kornell (2005), the tendency for learners to choose spaced practice over massed practice should decline as item difficulty becomes greater. Support originally was obtained when pairs containing unfamiliar…

Theories of self-regulated study assume that learners monitor item difficulty when making decisions about which items to select for study. To complement such theories, the authors propose an agenda-based regulation (ABR) model in which learners' study decisions are guided by an agenda that learners develop to prioritize items for study, given…