Introduction: Item response theory (IRT) has received much attention in validation of assessment instrument because it allows the estimation of students' ability from any set of the items. Item response theory allows the difficulty and discrimination levels of each item on the test to be estimated. In the framework of IRT, item characteristics are…

We revisit the empirically observed positive correlation between DIF and difficulty studied by Freedle and commonly seen in tests of verbal proficiency when comparing populations of different mean latent proficiency levels. It is shown that a positive correlation between DIF and difficulty estimates is actually an expected result (absent any true…

Full-information item analysis provides item developers and reviewers comprehensive empirical evidence of item quality, including option response frequency, point-biserial index (PBI) for distractors, mean-scores of respondents selecting each option, and option trace lines. The multi-serial index (MSI) is introduced as a more informative…

Surveys with rating scales are often used in higher education research to measure student learning and development, yet testing and reporting on the longitudinal psychometric properties of these instruments is rare. Rasch techniques allow scholars to map item difficulty and individual aptitude on the same linear, continuous scale to compare…

Classroom assessment research in the United States has shifted away from the examination of teacher-made tests, but such tests are still widely used and have an enormous impact on students' educational experiences. Given the major shifts in educational policy in the United States, including the widespread adoption of the Common Core State…

In this work, we consider learning analytics for primary and secondary schools from the perspective of the designer of a learning system. We provide an overview of practically useful analytics techniques with descriptions of their applications and specific illustrations. We highlight data biases and caveats that complicate the analysis and its…

In 2013, the Council of Chief State School Officers (CCSSO), working collaboratively with state education agencies, released a set of criteria for states to use to evaluate and procure high-quality assessments. The mathematics section of the document included five content-specific criteria to evaluate alignment of assessments to college- and…

In a signal detection theory (SDT) approach to multiple choice exams, examinees are viewed as choosing, for each item, the alternative that is perceived as being the most plausible, with perceived plausibility depending in part on whether or not an item is known. The SDT model is a process model and provides measures of item difficulty, item…

The transition of our large ([approximately]300 student) introductory soil science course to the online setting created several challenges, including engaging first- and second-year students, providing meaningful hands-on learning activities, and setting up online exams. The objective of this paper is to describe the development and use of…

A common approach to increase test security in higher educational high-stakes testing is the use of different test forms with identical items but different item orders. The effects of such varied item orders are relatively well studied, but findings have generally been mixed. When multiple test forms with different item orders are used, we argue…

Since its implementation in 2004, the Canadian English Language Benchmark Assessment for Nurses (CELBAN) has been accepted as evidence of language ability for licensure of internationally educated nurses (IENs) in Canada. This article focuses on the complexities of sustaining an occupation-specific assessment over time. The authors reference the…

The choice of anchor tests is crucial in applications of the nonequivalent groups with anchor test design of equating. Sinharay and Holland (2006, 2007) suggested "miditests," which are anchor tests that are content-representative and have the same mean item difficulty as the total test but have a smaller spread of item difficulties.…

This article describes how with the advent of computer-based technology, there is movement from manual to automated systems for different aspects of the education system. Testing is an essential part of teaching process that helps academics in classifying the level of students and evaluating the outcomes of their teaching process. The testing…

The concept of intrinsic complexity explains why different problems of the same type, tackled by the same problem solver, can require different times to solve and yield solutions of different quality. This paper proposes a general four-step approach that can be used to establish a model for the intrinsic complexity of a problem class in terms of…

Assessment is a key lever for educational improvement. Assessments can be used to monitor, signal, and influence science teaching and learning -- provided that they are of high quality, reflect the rigor and intent of academic standards, and elicit meaningful student performances. Since the release of "A Framework for K-12 Science…