The purpose of this paper is to provide states and school districts with processes they can use to help ensure high-quality data collection during teacher observations. Educational Testing Service's (ETS's) goal in writing it is to share the knowledge and expertise they gained: (1) from designing and implementing scoring processes for the Measures…

The study considers the assessment of L2 English learners by trained raters in paired and group oral assessments in comparison to an individual, monologue assessment, to determine 1) the degree to which raters assign pairs/groups shared (the same) scores and the degree to which raters give individual members of pairs/groups higher or lower as…

The authors address the reliability of scores obtained on the summative performance assessments during the pilot year of our research. Contrary to classical test theory, we discussed the advantages of using generalizability theory for estimating reliability of scores for summative performance assessments. Generalizability theory was used as the…

In 2009, the New Teacher Project's "The Widget Effect" documented the failure of U.S. public school districts to recognize and act on differences in teacher effectiveness. We revisit these findings by compiling teacher performance ratings across 24 states that adopted major reforms to their teacher evaluation systems. In the vast…

This issue of "Assessment Update" presents the following articles: (1) Lessons Learned: Graduate Student Assessment (Michael Cherry and Lesley Page); (2) Assessing the Assessor: A Response to Fulcher and Bashkov's "Do We Practice What We Preach? The Accountability of an Assessment Office" (Christine Robinson, John G. M.…

In the article, "What examiners do: What thesis students should know", we identified 11 things that thesis examiners do as they read and judge a thesis. But, we left a gap in the research: knowing this, What should thesis students do to write for their examiners? In this article, I fill the gap. The advice for thesis students is: first,…

Background: Marking of essays is mainly carried out by human raters who bring in their own subjective and idiosyncratic evaluation criteria, which sometimes lead to discrepancy. This discrepancy may in turn raise issues like reliability and fairness. The current research attempts to explore the evaluation criteria of markers on a national level…

Correct determination of whether or not the objectives are achieved or of the level of achievement of the objectives is vital in educational organizations. In this context, one of the indicators of how teachers serve the organizational objectives is performance evaluation. In Turkish Educational System, the evaluation of the performance of…

This study develops a model of analytic rating scales to assess L2 Chinese oral performance. It uses Exploratory Factor Analysis (EFA) to identify a model and employs Confirmative Factor Analysis (CFA) in a separate dataset to test the degree of model fit. The researcher videotaped ten speeches and ACTFL professional raters assessed the oral…

When good model-data fit is observed, the Many-Facet Rasch (MFR) model acts as a linking and equating model that can be used to estimate student achievement, item difficulties, and rater severity on the same linear continuum. Given sufficient connectivity among the facets, the MFR model provides estimates of student achievement that are equated to…

This study employs longitudinal Conversation Analysis (CA) to examine one TA's follow-up contributions in a series of EFL group discussion tests. By tracking the TA's interactional practices across 18 groups, we observe how she adapts her turn design by increasingly aligning towards that of the novice English speakers. The TA initially attempts to…

Throughout the dissertation process, the chair and committee members provide feedback regarding quality to help the doctoral candidate to produce the highest-quality document and become an independent scholar. Nevertheless, results of previous research suggest that overall dissertation quality generally is poor. Because much of the feedback about…

Amongst extensive research on thesis writing quality, few provide a detailed account of common writing challenges/errors as perceived by examiners vs. students. In this study, eight recent English theses marked-up by examiners and defended in Iranian universities were selected randomly among those thesis made available (n=45). Based on an…

What is--or should be--the purpose of a teacher evaluation system? To compare teachers? To inform the public? To help teachers improve? Legislators might say that a teacher evaluation system's purpose is to remove bad teachers. Administrators might say that it is to assist them in matching teacher skills to specific groups of students. Parents…

Professional school counselors have been advocating for their role as counselors in the schools for decades (Galassi & Akos, 2007; Gysbers, 2002; Slaten & Baskin, 2013). Although researchers have addressed this concern through advocacy in service and writing, school counselors continue to perform a significant amount of non-counseling…