The article addresses some problems and options in setting standards on language tests and examinations. More specifically, it reports on a set of three workshops conducted in the European context where standard setting in language education typically concerns linking tests and examinations to the Council of Europe's "Common European…

Assessors' scores in performance assessments are known to be highly variable. Attempted improvements through training or rating format have achieved minimal gains. The mechanisms that contribute to variability in assessors' scoring remain unclear. This study investigated these mechanisms. We used a qualitative approach to study…

In recognition of the rating scale as a crucial tool of performance assessment, this study aims to establish a rating scale suitable for a Story Retelling Speaking Test (SRST), which is a semidirect test of speaking ability in English as a foreign language for classroom use. To identify an appropriate scale, three rating scales, all of which have…

The current study examined the role of increased attentional load in 24 month-old children's multistep problem-solving behavior. Children solved an object-based nonspatial working-memory search task, to which a motor component of varying difficulty was added. Significant disruptions in search performance were observed with the introduction of the…

The study investigated the effects of the Cognitive Level and Quality Writing Assessment (CLAQWA) rubric on the cognitive and writing skill growth in freshmen composition classes. The participants were enrolled at a Midwestern state university. The nonequivalent control group design used quantitative analysis with selected criteria from the CLAQWA…

In 2004 the General Assembly of the State of Ohio (USA) enacted Senate Bill 2, which created the Educator Standards Board "to develop and recommend to the state board of education standards for entering and continuing in the teaching and principalship professions". Since that time the Ohio Department of Education has established a…

The current study evaluated the accuracy of the Structured Interview of Reported Symptoms, Second Edition (SIRS-2) in a criterion-group study using a sample of forensic psychiatric patients and a community simulation sample, comparing it to the original SIRS and to results published in the SIRS-2 manual. The SIRS-2 yielded an impressive…

Consistency in the title, problem, purpose, and research question improve the logic and transparency of research. When these components of research are aligned research design and planning are more coherent and research reports are more readable. This article reviews the process for checking for and improving consistency. Numerous examples of…

States in the midst of implementing evaluation reforms face a common dilemma: how to ensure that all your districts are implementing quality educator evaluation systems while still providing them with the flexibility to design systems best suited to their own unique needs. One answer is to create an equivalency process (also known as an approval…

Samejima hypothesized that non-monotonically increasing item response functions (IRFs) of ability might occur for multiple-choice items (referred to here as "Samejima items") if low ability test takers with some, though incomplete, knowledge or skill are drawn to a particularly attractive distractor, while very low ability test takers…

1.2 billion users in Facebook, 17 million articles in Wikipedia, and 190 million tweets per day have demanded significant increase of information processing through Internet in recent years. Similarly life sciences and bioinformatics also have faced issues of processing Big data due to the explosion of publicly available genomic information…

This quantitative study explored if significant differences exist between how fifth-grade students produce a written response to a narrative prompt using online versus offline writing platforms. The cultural and social trend of instructional and assessment writing paradigms in education is shifting to online writing platforms (National Assessment…

In this paper, we provide an overview of psychometric procedures and guidelines Educational Testing Service (ETS) uses to evaluate automated essay scoring for operational use. We briefly describe the e-rater system, the procedures and criteria used to evaluate e-rater, implications for a range of potential uses of e-rater, and directions for…

This paper analyses the responses of 247 middle school students to items requiring the concept of average in three different contexts: a city's weather reported in maximum daily temperature, the number of children in a family, and the price of houses. The mixed but overall disappointing performance on the six items in the three contexts indicates…

The consequences associated with the uses and interpretations of scores for many credentialing testing programs have important implications for a range of stakeholders. Within licensure settings specifically, results from examination programs are often one of the final steps in the process of assessing whether individuals will be allowed to enter…