The Mastery Performance-Goal Orientation Test (MP-GOT) assesses performance- and mastery-orientation along a single bipolar dimension. In Study 1 (n = 198) we explored its psychometric properties and determined whether its results were related to learning. In addition, we employed Latent Profile Analysis to explore goal orientation profiles. In…

In the context of digital technologies, computational thinking (CT) is considered a key competence of primary school teachers. However, assessment tools to measure in-service teachers' CT competence are rare. The current study explains the design, development, and validation of a scale to determine the in-service teachers' mastery level of their…

To address the computational constraints of parameter estimation in the polytomous Cognitive Diagnosis Model (pCDM) in large-scale high data volume situations, this study proposes two two-stage polytomous attribute estimation methods: P_max and P_linear. The effects of the two-stage methods were studied via a Monte Carlo simulation study, and the…

This study details the construction of parameters for generating subskill mastery math measures to be used for screening, intervention planning, progress monitoring, and proximal program evaluation. Parameters for generating assessment measures were built and tested to verify initial equivalence of generated measures using potential digits correct…

These proceedings contain the papers of the 18th International Conference on Cognition and Exploratory Learning in the Digital Age (CELDA 2021), held virtually, due to an exceptional situation caused by the COVID-19 pandemic, from October 13-15, 2021, and organized by the International Association for Development of the Information Society…

Current methods for obtaining reliability indices for mastery tests can be laborious. This paper offers practitioners tables from which agreement and kappa coefficients can be read directly and provides criterion for acceptable values of agreement and kappa coefficients. (TJH)

An index for the dependability of mastery tests is described. Assumptions necessary for the index and the mathematical development of the index are provided. (Author/JKS)

Many mastery decisions are made based on assessment devices that are known to be less than perfectly reliable. A significant portion of examinees who have scores near the cutting score risk misclassification, even for instruments with relatively high retest reliability. Suggestions for minimizing this risk are offered. (Author/MT)

Two indices for assessing the efficiency of decisions in mastery testing are proposed. The indices are generalizations of the raw agreement index and the kappa index. Empirical examples of these indices are given. (Author/JKS)

Simulated data based on five test score distributions indicate that a slight modification of the asymptotic normal theory for the estimation of the p and kappa indices in mastery testing will provide results which are in close agreement with those based on small samples from the beta-binomial distribution. (Author/BW)

The dependence of reliability indices for mastery tests on mean and cutoff scores was examined in the case of three decision-theoretic indices. Dependence of kappa on mean and cutoff scores was opposite to that of the proportion of correct decisions, which was linearly related to average threshold loss. (Author/BW)

The classical estimate of a binomial probability function is to estimate its mean in the usual manner and to substitute the results in the appropriate expression. Two alternative estimation procedures are described and examined. Emphasis is given to the single administration estimate of the mastery test reliability. (Author/CTM)

A number of different reliability coefficients have recently been proposed for tests used to differentiate between groups such as masters and nonmasters. One promising index is the proportion of students in a class that are consistently assigned to the same mastery group across two testings. The present paper proposes a single test administration…

The use of Cohen's kappa index as a measure of the reliability of multiple classifications is developed. Special cases of the index as well as the effects of test length on the index are also explored. (JKS)

Huynh (1976) suggested a method of approximating the reliability coefficient of a mastery test. The present study examines the accuracy of Huynh's approximation and also describes a computationally simpler approximation which appears to be generally more accurate than the former. (Author/RL)