Government entities set criteria for institutions that have teacher educator programs to use online assessment tools to show continuous ongoing evaluation, and use data from the tools to guide the improvement of courses. The purpose of this qualitative, multi-case study was to discover how Instructional Designers-by-Assignment (IDBA) are using…

Educators, parents, and the public depend on accurate, valid, reliable, and timely information about student academic performance. Testing irregularities--breaches of test security or improper administration of academic testing--undermine efforts to use those data to improve student achievement. Unfortunately, there have been high-profile and…

We have been developing a digital concept maps website (www.conceptmapsforlearning.com) based on the principles of effective assessment for learning. The purpose of this paper is to reveal its promising contributions to formative evaluation practices. The website reduces the workload of teachers as well as provides immediate and delayed feedback…

In this paper, we provide an overview of psychometric procedures and guidelines Educational Testing Service (ETS) uses to evaluate automated essay scoring for operational use. We briefly describe the e-rater system, the procedures and criteria used to evaluate e-rater, implications for a range of potential uses of e-rater, and directions for…

The Fall 2007 and Spring 2008 pilot tests for the "CBAL"™ Writing assessment included experimental keystroke logging capabilities. This report documents the approaches used to capture the keystroke logs and the algorithms used to process the outputs. It also includes some preliminary findings based on the pilot data. In particular, it…

The purposes of this study are twofold. First, to investigate the construct or factorial structure of a set of Reading and Mathematics computerized adaptive tests (CAT), "Measures of Academic Progress" (MAP), given in different states at different grades and academic terms. The second purpose is to investigate the invariance of test…

Computer-based tests (CBTs) often use random ordering of items in order to minimize item exposure and reduce the potential for answer copying. Little research has been done, however, to examine item position effects for these tests. In this study, different versions of a Rasch model and different response time models were examined and applied to…

This study explored the impact of partial credit scoring of one type of innovative items (multiple-response items) in a computerized adaptive version of a large-scale licensure pretest and operational test settings. The impacts of partial credit scoring on the estimation of the ability parameters and classification decisions in operational test…

In the human sciences, a common assumption is that latent traits have a hierarchical structure. Higher order item response theory models have been developed to account for this hierarchy. In this study, computerized adaptive testing (CAT) algorithms based on these kinds of models were implemented, and their performance under a variety of…

Traditional methods for item selection in computerized adaptive testing only focus on item information without taking into consideration the time required to answer an item. As a result, some examinees may receive a set of items that take a very long time to finish, and information is not accrued as efficiently as possible. The authors propose two…

This study introduces emotional feedback as a construct in an acceptance model. It explores the effect of emotional feedback on behavioral intention to use Computer Based Assessment (CBA). A female Embodied Conversational Agent (ECA) with empathetic encouragement behavior was displayed as emotional feedback. More specifically, this research aims…

The ultimate aim of our research project was to use the Google Web Speech API to automate scoring of elicited imitation (EI) tests. However, in order to achieve this goal, we had to take a number of preparatory steps. We needed to assess how accurate this speech recognition tool is in recognizing native speakers' production of the test items; we…

Many postsecondary institutions, and community colleges in particular, require that students demonstrate specified levels of literacy and numeracy before taking college-level courses. Typically, students have been assessed using two widely available tests--ACCUPLACER and Compass. However, placement testing practice is beginning to change for three…

Assessment is a delicate task in the overall teaching process because it may require significant time and may be prone to subjectivity. Subjectivity is especially true for disciplines in which perceptual factors play a key role in the evaluation. In previous decades, computer-based assessment techniques were developed to address the…

Adaptive testing is advantageous in that it provides more efficient ability estimates with fewer items than linear testing does. Item-driven adaptive pretesting may also offer similar advantages, and verification of such a hypothesis about item calibration was the main objective of this study. A suitability index (SI) was introduced to adaptively…