Item response theory parameters have to be estimated, and because of the estimation process, they do have uncertainty in them. In most large-scale testing programs, the parameters are stored in item banks, and automated test assembly algorithms are applied to assemble operational test forms. These algorithms treat item parameters as fixed values,…

The random-threshold generalized unfolding model (RTGUM) was developed by treating the thresholds in the generalized unfolding model as random effects rather than fixed effects to account for the subjective nature of the selection of categories in Likert items. The parameters of the new model can be estimated with the JAGS (Just Another Gibbs…

A new score report based on a mechanism of formative assessment and feedback is developed to offer individual testees not only their final scores but also their sub-scale scores, their percentile position, as well as corresponding feedback on self-regulation strategies. Structural equation modeling is adopted in the confirmatory factor analysis to…

Through simulated data, five multidimensional computerized adaptive testing (MCAT) selection procedures with varying test lengths are examined and compared using different stopping rules. Fixed item exposure rates are used for all the items, and the Priority Index (PI) method is used for the content constraints. Two stopping rules, standard error…

The Internet has revolutionized the way users share and acquire knowledge. As important and popular Web-based applications, online discussion forums provide interactive platforms for users to exchange information and report problems. With the rapid growth of social networks and an ever increasing number of Internet users, online forums have…

A system was developed for automatic generation of problems and solutions for examinations in a university distance course in discrete mathematics and tested in a pilot experiment involving 200 students. Considering the success of such systems in the past, particularly including automatic assessment, it should not take long before such systems are…

In some popular test designs (including computerized adaptive testing and multistage testing), many item pairs are not administered to any test takers, which may result in some complications during dimensionality analyses. In this paper, a modified DETECT index is proposed in order to perform dimensionality analyses for response data from such…

We investigated students' metacognitive experiences with regard to feelings of difficulty (FD), feelings of satisfaction (FS), and estimate of effort (EE), employing either computerized adaptive testing (CAT) or computerized fixed item testing (FIT). In an experimental approach, 174 students in grades 10 to 13 were tested either with a CAT or a…

The Trail-Making Test (TMT) is a widely used neuropsychological measure that assesses visuomotor abilities and cognitive flexibility. For the TMT-A condition participants are required to locate and connect numbers (i.e. 1-2-3...) while in the TMT-B condition participants perform the set-shifting task of locating and connecting numbers and letters…

Advances in technology and changes in educational strategies have resulted in the integration of technology in the classroom. Multimedia instructional tools (MMIT) provide student-centered active-learning instructional activities. MMITs are common in introductory computer applications courses based on the premise that MMITs should increase student…

In this paper we discuss the use of automated assessment in a variety of computer science courses that have been taught at Israel Academic College by the authors. The course assignments were assessed entirely automatically using Checkpoint, a web-based automated assessment framework. The assignments all used free-text questions (where the students…

This study reports 2 eye-tracking experiments investigating form interference during sentence-level silent reading. The items involved reduced and unreduced relative clauses (RCs) with words that were orthographically and phonologically similar "(injection-infection"; O+P+, Experiment 1) as well as with words that were orthographically…

This case deals with the redesign of a standard telecourse--printed material, professional studio video recordings and phone tutoring--into an online course. The redesign involved an adjunct professor in the Humanities having some experience in distance education but little with learning technologies. It was a two-year project including the grant…

The present study challenges the validity of elicited imitation (EI) as a measure for implicit knowledge, investigating to what extent online error detection and subsequent sentence repetition draw on implicit knowledge. To assess online detection during listening, a word monitoring component was built into an EI task. Advanced-level Japanese L2…

Korean has visually salient syllable units that are often mapped onto either prefixes or suffixes in derived words. In addition, prefixed and suffixed words may be processed differently given a left-to-right parsing procedure and the need to resolve morphemic ambiguity in prefixes in Korean. To test this hypothesis, four experiments using the…