Nearly all private, government and non-governmental organizations that receive government funding to run social or health promotion programs in the United States are required to conduct program evaluations and to report findings to the funding agency. Reports are usually due at the end of a funding cycle and they may or may not have an influence…

Along with a more humanitarian movement in language testing, accountability to contextual variables in the design and development of any assessment enterprise is emphasized. However, when it comes to writing assessment, it is found that multiplicity of rating scales developed to fit diverse contexts is mainly headed by well-known native testing…

Peer review of teachers is controversial for several reasons. Some say peer reviewers encroach on the rightful domain of the principal as instructional leader. Others argue that, because peer evaluators are fellow teachers, they may be biased or unwilling to make hard decisions. Many teachers find the prospect of peer evaluation unsettling because…

Many telephone surveys require interviewers to observe and record respondents' gender based solely on respondents' voice. Researchers may rely on these observations to: (1) screen for study eligibility; (2) determine skip patterns; (3) foster interviewer tailoring strategies; (4) contribute to nonresponse assessment and adjustments; (5)…

The purpose of this paper is to share a description of the process used by Fort Hays State University (FHSU) as a self-study of the FHSU alternative certification program, known as Transition to Teaching. Team members used the Quality Indicators designed as a part of a Department of Education Transition to Teaching Grant called the KNOTtT Project.…

Paired speaking tests are now commonly used in both high-stakes testing and classroom assessment contexts. The co-construction of discourse by candidates is regarded as a strength of paired speaking tests, as candidates have the opportunity to display a wider range of interactional competencies, including turn taking, initiating topics, and…

Major advances in the number, capabilities, and quality of state, national, and transnational databases have opened up new opportunities for evaluators. Both large-scale data sets collected for administrative purposes and those collected by other researchers can provide data for a variety of evaluation-related activities. These include (a)…

This paper describes one reviewer's experience of reviewing for the journal "System" over an eight-year period, 2003-2011. The paper reports on the reviews produced by the single reviewer, which have been compiled into a specific purpose--an "occluded"--corpus (Swales, 1996) of 122 reviews, comprising 93,000 words. The paper first describes the…

Prior research indicates that the overall reliability of performance ratings can be improved by using ordinary least squares (OLS) regression to adjust for rater effects. The present investigation extends previous work by evaluating the impact of OLS adjustment on standard errors of measurement ("SEM") at specific score levels. In…

Language assessment courses (LACs) are taught by professionals who have majored in the area of language testing (language testers or LTs), but also by others who come from different language-related majors (non-language testers, non-LTs). Different language assessment courses may be developed, depending on who teaches the course and the…

This article focuses on the impact of a specific formative assessment procedure, negotiated assessment, on teacher professional learning. Negotiations between the assessor and the teacher as assessee seem to be especially promising for this teacher learning. However, there is no empirical evidence yet that has confirmed this. We explored teachers'…

Many teachers are already keen observers and skilled in supporting and collaborating with their colleagues. Leveraging this rich talent among staff can be an efficient way to address capacity challenges and enrich teachers' evaluations with more targeted feedback. Teachers, however, require training to become systematic, reliable observers who can…

Prior learning assessment (PLA) is a non-traditional component of higher education that recognizes that life experiences and learning from outside of academia can be deemed valid and awarded college credit. This qualitative study explored the perceptions of faculty assessors of PLA portfolios. This study sought to understand, through a…

This research investigated the relation between rater cognition and rater behavior, focusing on differential severity/leniency regarding criteria. Participants comprised a sample of 18 raters examined in a previous rater cognition study (Eckes, 2008b). These raters, who were known to differ widely in their perceptions of criterion importance,…

An aspect of assessment which has received little attention compared with perennial concerns, such as standards or reliability, is the role of judgment in marking. This paper explores marking as an act of judgment, paying particular attention to the nature of judgment and the processes involved. It brings together studies which have explored…