The mentoring and evaluating process enhances quality of teachers' serving in early childhood programs. The Early Educator Support (EES) Program is a new model of mentoring and coaching that aims to lend support to PreK teachers' working in nonpublic school settings. In this qualitative study, we examined the lived experiences of 12 mentors and…

This study examines the feasibility and potential advantages of using large language models, in particular GPT-4o, to perform partial credit grading of large numbers of student written responses to introductory level physics problems. Students were instructed to write down verbal explanations of their reasoning process when solving one conceptual…

This study's purpose is to extend our understanding of school leadership for student learning by identifying high-leverage teacher evaluation practices that improve teaching. A partnership with a state education agency administered a teacher questionnaire regarding evaluation practices multiple times in one semester, then linked teacher responses…

In 2004, Robert Stake published a provocative article that asked: "How Far Dare an Evaluator Go Toward Saving the World?" This question raises the issue of what role evaluators' values play in the conduct of evaluations. Following review of Stake's premises, I present value statements from a diverse group of 40 evaluators working with…

This controlled experimental study investigated the interaction of variables associated with rating the pronunciation component of high-stakes English-language-speaking tests such as IELTS and TOEFL iBT. One hundred experienced raters who were all either familiar or unfamiliar with Brazilian-accented English or Papua New Guinean Tok Pisin-accented…

The current paper intends to exploit the Many Facet Rasch Model to investigate and compare the impact of situations (items) and raters on test takers' performance on the Written Discourse Completion Test (WDCT) and Discourse Self-Assessment Tests (DSAT). In this study, the participants were 110 English as a Foreign Language (EFL) students at…

Evaluating the impact of Academic Literacies teaching (Lea and Street [1998. "Student Writing in Higher Education: An Academic Literacies Approach." "Studies in Higher Education" 23 (2): 157-72. doi:10.1080/03075079812331380364]) is difficult, as it involves gauging whether writers: (1) gain better understanding of what…

This study aims to examine whether differences exist in the factors influencing the difficulty of scoring English summaries and determining scores based on the raters' attributes, and to collect candid opinions, considerations, and tentative suggestions for future improvements to the analytic rubric of summary writing for English learners. In this…

Teacher performance scores inform education leaders' management of teacher human resources. However, prior research has implied that different interpretations of performance criteria between teachers and their evaluators suppress teacher development. Although research has examined teacher perceptions of performance scores and compared teacher…

In this Brief Report, we describe an evaluation of and revisions to a rubric adapted from the Jacobs et al.'s (1981) ESL COMPOSITION PROFILE, with four rubric categories and 20-point rating scales, in the context of an intensive English program writing placement test. Analysis of 4 years of rating data (2016-2021, including 434 essays) using…

Modern-day educators often share a sense they lack voice and agency with school administration. A classic example is curriculum development, where third-party designers develop uncontextualized curricula, and teachers then must implement the design even if inefficient and ineffective. What happens if this identical situation occurs at the program…

Human Resource (HR) practices like performance appraisal (PA) training are meant to ensure that employees are equipped with the knowledge and skills needed for the attainment of organisational goals. However, gaps still exist on the relationship between PA employees' training and employees' performance. This study aimed to establish the…

Summary writing is essential for academic success, and has attracted renewed interest in academic research and large-scale language test. However, less attention has been paid to the development and evaluation of the scoring scales of summary writing. This study reports on the validation of a summary rubric that represented an approach to scale…

Synthesis writing is widely taught across domains and serves as an important means of assessing writing ability, text comprehension, and content learning. Synthesis writing differs from other types of writing in terms of both cognitive and task demands because it requires writers to integrate information across source materials. However, little is…

This article presents a new tool called Critical Evaluation Capital (CEC) designed to address issues of equity and social justice in program evaluation. CEC is grounded in the tenants of critical race theory and inspired by Yosso's work on community cultural wealth which raises critical issues of positionality and access. CEC is a system for…