In this paper, we provide an overview of psychometric procedures and guidelines Educational Testing Service (ETS) uses to evaluate automated essay scoring for operational use. We briefly describe the e-rater system, the procedures and criteria used to evaluate e-rater, implications for a range of potential uses of e-rater, and directions for…

This paper describes the theoretical basis and development of the iSELF: an Internet-tool for Self-Evaluation and Learner Feedback to stimulate self-directed learning in ubiquitous learning environments. In ubiquitous learning, learners follow their own trails of interest, scaffolded by coaches, peers and tools for thinking and learning.…

Effective and efficient measurement of the development of skill and knowledge, especially in domains of human activity that involve complex and challenging problems, is important with regard to workplace and academic performance. However, there has been little progress in the area of practical measurement and assessment, due in part to the lack of…

This article presents key steps in the design and analysis of a computer based problem-solving assessment featuring interactive tasks. The purpose of the assessment is to support targeted instruction for students by diagnosing strengths and weaknesses at different stages of problem-solving. The first focus of this article is the task piloting…

This session will describe the social validity, usability and intent to use of an interactive, state-of-the-art, professional development program based on the Response to Intervention (RtI) approach and its core components (e.g., problem-solving strategy; three tiers of intervention service delivery with universal, selected and intensive…

Purpose: The purpose of this paper is to exhibit the integration of learning and study strategies inventory (LASSI) with the City University of Hong Kong information systems to promote teaching and learning within the university. Design/methodology/approach: From the 2006 entry cohort, all undergraduate freshmen at City University of Hong Kong are…

Purpose: The purpose of this paper is to describe how and why Texas A&M University at Qatar (TAMUQ) has developed a system aiming to effectively place students in freshman and developmental English programs. The placement system includes: triangulating data from external test scores, with scores from a panel-marked hand-written essay (HWE),…

The Bologna Declaration brought reforms into higher education that imply changes in teaching methods, didactic materials and textbooks, infrastructures and laboratories, etc. Statistics and mathematics are disciplines that traditionally have the worst success rates, particularly in non-mathematics core curricula courses. This research project,…

Purpose: The benefits of an educational shift to graduate attribute development have been foregrounded in the educational literature since the early 1990s. Attribute mapping in documentation with no change to assessment constitutes a surface approach. This paper aims to use as an example a deep approach that uses software to facilitate staff and…

This article argues that if multimodal and new literacies are to become common practices in schools, they have to be included in both school and provincial/state large-scale assessment programmes. Building on current criterion-referenced testing in Newfoundland and Labrador which assesses a range of literacies (viewing, reading, writing,…

Latent semantic analysis (LSA) is an automated, statistical technique for comparing the semantic similarity of words or documents. In this article, I examine the application of LSA to automated essay scoring. I compare LSA methods to earlier statistical methods for assessing essay quality, and critically review contemporary essay-scoring systems…

Since 2002, Project WHIRL (Wireless Handhelds In Reflection on Learning) has investigated potential uses of handheld computers in K-12 science classrooms using a teacher-involved process of software development and field trials. Te project is a three-year research and development grant from the National Science Foundation, and it is a partnership…

This keynote address reported on the current state of science education in three areas: curriculum development, attitudes toward science, and assessment in science. Three major projects in science curriculum reform were the focus of the section on curriculum development: (1) Project 2061; (2) Scope, Sequence, and Coordination; and (3) Earth…