For efficiency, Computerized Adaptive Test (CAT) algorithm selects items with the maximum information, typically with a 50% probability of being answered correctly. However, examinees may not be satisfied if they only correctly answer 50% of the items. Researchers discovered that changing the item selection algorithms to choose easier items (i.e.,…

This article introduces Xiao-Shih, the first intelligent question answering bot on Chinese-based massive open online courses (MOOCs). Question answering is critical for solving individual problems. However, instructors on MOOCs must respond to many questions, and learners must wait a long time for answers. To address this issue, Xiao-Shih…

Access to high-quality education at scale is limited by the difficulty of providing student feedback on open-ended assignments in structured domains like programming, graphics, and short response questions. This problem has proven to be exceptionally difficult: for humans, it requires large amounts of manual work, and for computers, until…

Purpose: Based on the general formula, which depends on the length and difficulty of the test, the number of respondents and the number of ability levels, this study aims to provide a closed formula for the adaptive tests with medium difficulty (probability of solution is p = 1/2) to determine the accuracy of the parameters for each item and in…

Administrative problems such as computer malfunction and power outage occasionally lead to missing item scores and hence to incomplete data on mastery tests such as the AP and U.S. Medical Licensing examinations. Investigators are often interested in estimating the probabilities of passing of the examinees with incomplete data on mastery tests.…

Many studies have shown that using a computer-aided detection (CAD) system does not significantly improve diagnostic accuracy in radiology, possibly because radiologists fail to interpret the CAD results properly. We tested this possibility using screening mammography as an illustrative example. We carried out two experiments, one using 28…

A large-scale simulation study of the assessment effectiveness of a particular instantiation of knowledge space theory is described. In this study, data from more than 700,000 actual assessments in mathematics using the ALEKS (Assessment and LEarning in Knowledge Spaces) software were used to determine response probabilities for the same number of…

Can a single adjective immediately influence message-building during sentence processing? We presented participants with 168 sentence contexts, such as "His skin was red from spending the day at the …" Sentences ended with either the most expected word ("beach") or a low cloze probability completion ("pool"). Nouns…

In learning, errors are ubiquitous and inevitable. As these errors may signal otherwise latent cognitive processes, tutors--and students alike--can greatly benefit from the information they provide. In this paper, we introduce and evaluate the Systematic Error Tracing (SET) model that identifies the possible causes of systematically observed…

This paper describes the development and efficacy of an online tool for assessing the numeracy of undergraduate students. The tool was designed to be easy to administer, provide immediate feedback to students on whether they had the required level of numeracy, and to be consistent with other measures of adult numeracy. When used with students…

Frequent forced switching between tasks has been shown to reduce switch costs and increase voluntary switch rates. So far, however, the boundary conditions of the influence of forced task switching on voluntary task switching are unknown. Thus, the present study was aimed to test different aspects of generalizability (across items, tasks, and…

A sample of 327 engineering bachelor students from a public university in Mexico took part in an information integration study to explore systematic thinking underlying propensity for cheating during a course exam. All study participants were provided with written descriptions of 12 scenarios pertaining to the academic evaluation criteria and were…

The development of cognitive diagnostic-computerized adaptive testing (CD-CAT) has provided a new perspective for gaining information about examinees' mastery on a set of cognitive attributes. This study proposes a new item selection method within the framework of dual-objective CD-CAT that simultaneously addresses examinees' attribute mastery…

Due to increased use of computer-based assessments, comparability studies are moving beyond paper-and-pencil versus computer-based assessments to analyze variances with computers. It is therefore practically important to determine whether screen size and definition of the device affect students' performance. Using data from a large school district…

In the machine learning literature, it is commonly accepted as fact that as calibration sample sizes increase, Naïve Bayes classifiers initially outperform Logistic Regression classifiers in terms of classification accuracy. Applied to subtests from an on-line final examination and from a highly regarded certification examination, this study shows…