This article illustrates five different methods for estimating Angoff cut scores using item response theory (IRT) models. These include maximum likelihood (ML), expected a priori (EAP), modal a priori (MAP), and weighted maximum likelihood (WML) estimators, as well as the most commonly used approach based on translating ratings through the test…

This study examined changes in China's college student fitness test batteries since its inception in 1954. Using the constant content comparison method, the testing components, testing items and related cut-off values, testing methods, testing results utility, and testing material distribution were examined to identify the salient trends. The…

The present study investigates the best factor for controlling the item difficulty of multiple-choice English vocabulary questions generated by an automatic question generation system. Three factors are considered for controlling item difficulty: (1) reading passage difficulty, (2) semantic similarity between the correct answer and distractors,…

Exams assessment is one of the most tedious work for university teachers all over the world. Multiple choice theses make exams assessment a little bit easier, but the teacher cannot prepare more than 3-4 variants; in this case, the possibility of students for cheating from one another becomes a risk for "objective assessment outcome." On…

This study uses the nominal response model to investigate the effects of extreme response styles. The Zung Self-Rating Anxiety Scale (SAS) is a commonly used scale for the identification of anxiety disorders. In some cases, the response options are not extreme, ranging from "A little of the time" to "Most of the time;" in other…

Large-scale assessments are supported by a large item pool. An important task in test development is to assign items into scales that measure different characteristics of individuals, and a popular approach is cluster analysis of items. Classical methods in cluster analysis, such as the hierarchical clustering, K-means method, and latent-class…

The maximum likelihood estimate (MLE) of the ability parameter of an item response theory model with known item parameters was proved to be asymptotically normally distributed under a set of regularity conditions for tests involving dichotomous items and a unidimensional ability parameter (Klauer, 1990; Lord, 1983). This article first considers…

The statistical analysis of answer changes (ACs) has uncovered multiple testing irregularities on large-scale assessments and is now routinely performed at testing organizations. However, AC data has an uncertainty caused by technological or human factors. Therefore, existing statistics (e.g., number of wrong-to-right ACs) used to detect examinees…

Polytomous items are typically described by multiple category-related parameters; situations, however, arise in which a single index is needed to describe an item's location along a latent trait continuum. Situations in which a single index would be needed include item selection in computerized adaptive testing or test assembly. Therefore single…

Angoff-based standard setting is widely used, especially for high-stakes licensure assessments. Nonetheless, some critics have claimed that the judgment task is too cognitively complex for panelists, whereas others have explicitly challenged the consistency in (replicability of) standard-setting outcomes. Evidence of consistency in item judgments…

This article examines whether the way that PISA models item outcomes in mathematics affects the validity of its country rankings. As an alternative to PISA methodology a two-parameter model is applied to PISA mathematics item data from Canada and Finland for the year 2012. In the estimation procedure item difficulty and dispersion parameters are…

After explicit instruction on how to give possible bidirectional (two-way) causality explanations for a correlation, 240 students from eight sections of social psychology and research methods courses wrote "reverse causality" explanations on various test questions, creating a total of 882 answers. Averaging across multiple graded…

This work describes a systematic review of the research on take-home exams in tertiary education. It was found that there is some disagreement in the community about the virtues of take-home exams but also a lot of agreement. It is concluded that take-home exams may be the preferred choice of assessment method on the higher taxonomy levels because…

Functional Behavior Assessment (FBA) and behavior interventions have been effective in the management of challenging behavior among children with developmental disabilities including autism spectrum disorders. Research suggests the need for valid measurement instruments for verifying, calibrating and scoring competence in FBA and behavior…

Clinical judgment has become an increasingly important aspect of modern health service professionals. To ensure public safety, licensure exams must go beyond assessing only knowledge and skills when evaluating entry-level professions to evaluating clinical judgment. This importance necessitates licensure and certification examinations in these…