We introduce a short answer scoring engine made up of an ensemble of deep neural networks and a Latent Semantic Analysis-based model to score short constructed responses for a large suite of questions from a national assessment program. We evaluate the performance of the engine and show that the engine achieves above-human-level performance on a…

Currently, Developmental scales for children aged 0-6 years are a particularly valuable resource for assessing developmental milestones in children. Most scales are developed based on a broad conceptual framework, and their metric validation is insufficient and of low quality. The aim of this systematic review is to analyse the psychometric…

Language test developers have a responsibility to ensure that their tests are accessible to test takers of various backgrounds and characteristics and also that they have the opportunity to perform to the best of their ability. This principle is widely recognised by educational and language testing associations in guidelines for the production and…

This study examined recent technological developments in cognitive assessment and how these developments impact children's test behavior. The study consisted of two groups: one tested with an iPad and another tested with the standard paper and pencil format of the Wechsler Intelligence Scale for Children (WISC-IV). Independent groups t tests…

This paper is the second in a series from Educational Testing Service (ETS) that conceptualizes next-generation English language proficiency (ELP) assessment systems for K-12 English learners (ELs) in the United States.The first paper articulated a high-level conceptualization of next-generation ELP assessment systems (Hauck, Wolf, & Mislevy,…

The present study examined growth in writing quality associated with feedback provided by an automated essay evaluation system called PEG Writing. Equal numbers of students with disabilities (SWD) and typically-developing students (TD) matched on prior writing achievement were sampled (n = 1196 total). Data from a subsample of students (n = 655)…

This paper is the first in a series from Educational Testing Service (ETS) concerning English language proficiency (ELP) assessments for K-12 English learners (ELs). The goal of this paper, and the series, is to present research-based ideas, principles, and recommendations for consideration by those who are conceptualizing, developing, and…

Different learning tools are available within virtual learning environments, including forums, quizzes, and ePortfolios. This article investigates perceptions of helpfulness and ease of use of these three tools, including how they are impacted by learner characteristics and what predicts frequency of use of each tool. Critically, the relationship…

Item positions in educational assessments are often randomized across students to prevent cheating. However, if altering item positions results in any significant impact on students' performance, it may threaten the validity of test scores. Two widely used approaches for detecting position effects -- logistic regression and hierarchical…

Access to education is one of the main human rights. Everyone should have access to education and be capable of benefiting from it. However there are a number who are excluded, not because of a lack of ability but simply because they have a disability or specific need which current education systems do not address. A learning system in which…

In this article, we describe current research findings on assessment accommodations and universal design within the context of emerging interactive digital assessment tasks that employ simulations such as in science, technology, engineering, and mathematics (STEM). STEM education in many classrooms now includes digitally based activities such as…

This study investigated accommodations for students without disabilities in order to fill critical gaps in both knowledge and practices in fields of special education and educational assessments. In Ontario, students at different grade levels were assessed by the provincial assessments developed by the Education Quality and Accountability Office…

In this brave new world of education that has so much technology in it, there are many questions educational leaders must consider: (1) How can educational leaders keep up with the changes in technologies with educational applications? (2) How can technologies be used to level the playing field for students with disabilities, English language…

This chapter addresses the psychometric challenges in assessing English language learners (ELLs) and students with disabilities (SWDs). The first section addresses some general considerations in the assessment of ELLs and SWDs, including the prevalence of ELLs and SWDs in the student population, federal and state legislation that requires the…

There has been an increased interest in developing computer-adaptive testing (CAT) and multistage assessments for K-12 accountability assessments. The move to adaptive testing has been met with some resistance by those in the field of special education who express concern about routing of students with divergent profiles (e.g., some students with…