This research report provides an overview of the R&D efforts at Educational Testing Service related to its capability for automated scoring of nonnative spontaneous speech with the "SpeechRater"? automated scoring service since its initial version was deployed in 2006. While most aspects of this R&D work have been published in…

This paper provides an empirical account of the complex nature of early childhood teachers' work with growing accountability demands for collecting, reporting, and using child assessment data in the U.S. context. Particularly, this paper focuses on preschool teachers' experience of changes in their roles and pedagogical work in implementing a…

Maljkovic and Nakayama (1994) found that pop-out search performance is more efficient when a singleton target feature repeats rather than switches from 1 trial to the next--an effect known as priming of pop-out (PoP). They also reported findings indicating that the PoP effect is strongly automatic, as it was unaffected by knowledge of the upcoming…

Normalization of design rules and guidelines in English language tests to help future instructors and assessment developers in the field of examination of English as a foreign language have not been defined sufficiently until present days. In fact, there are several experiences and investigations related to the user's experiences in the handling…

In this paper, we explore how assessment technologies can support college and university assessment processes at multiple levels. Our goal is to help you think through your institutional assessment culture and processes, so you can identify tools that support your institution's approach to assessment. Multiple software systems can offer…

Despite the increasing importance of language testing, technology design matters related to testing have hardly been addressed in academic research, yet not enough has been written on test-taking anxiety related to computerized speaking tests. Although students nowadays are digital natives, in our experience they seem to experience a significant…

Interest in and use of technology-enhanced items has increased over the past decade. Given the additional time required to administer many technology-enhanced items and the increased expense required to develop them, it is important for testing programs to consider the utility of technology-enhanced items. The Technology-Enhanced Item Utility…

Four item selection indexes with and without exposure control are evaluated and compared in multidimensional computerized adaptive testing (CAT). The four item selection indices are D-optimality, Posterior expectation Kullback-Leibler information (KLP), the minimized error variance of the linear combination score with equal weight (V1), and the…

The current study explores students' collaboration and problem solving (CPS) abilities using a human-to-agent (H-A) computer-based collaborative problem solving assessment. Five CPS assessment units with 76 conversation-based items were constructed using the PISA 2015 CPS framework. In the experiment, 53,855 ninth and tenth graders in Taiwan were…

The paper presents the results of systematic review of modern scientific publications devoted to the digitalization of education. A review of Russian and foreign studies allows us to conclude that there is a relationship between high academic performance of students and the use of digital technologies. Other advantages of digitalization are:…

Research indicates that the performance-gap between English Language Learners (ELLs) and their non-ELL peers is partly due to ELLs' difficulty in understanding assessment language. Accommodations have been shown to narrow this performance-gap, but many accommodations studies have not used a randomized design and are based on relatively small…

Integrated speaking tasks requiring test takers to read and/or listen to stimulus texts and to incorporate their content into oral performances are now used in large-scale, high-stakes tests, including the TOEFL iBT. These tasks require test takers to identify, select, and combine relevant source text information to recognize key relationships…

Through Many-Facet Rasch analysis, this study explores the rating differences between 1 computer automatic rater and 5 expert teacher raters on scoring 119 students in a computerized English listening-speaking test. Results indicate that both automatic and the teacher raters demonstrate good inter-rater reliability, though the automatic rater…

Throughout this study technology and especially mobile phones was utilized in EFL classrooms in order to see whether it can influence the process of vocabulary formative assessment and consequently improve vocabulary learning of Iranian pre-intermediate EFL learners or not. Two groups of pre-intermediate EFL learners participated in this study.…

Background: The importance of reading motivation has led to the development of a large number of self-report reading motivation measures; however, there is still a need for a usable measure of adolescent reading motivation that captures a large number of theoretically and empirically distinct constructs. Methods: The current paper details the…