We examined the reliability of grading master's theses at a New Zealand university, where a variant of the academic journal review system is employed. The overall correlation between the grades recommended by internal and external markers of master's theses in psychology and applied psychology at this university was 0.39, which is similar to that…

Even when raters demonstrate agreement in the use of a measure, limited score variability or violation of often-ignored statistical assumptions can result in lower reliability estimates than intuitively expected. This article uses data drawn from two randomized controlled trials of schema therapy and cognitive behavioral therapy for the treatment…

This paper aims to survey the current landscape of education policy evaluation across OECD countries and economies by examining recent trends and contextual factors that can promote more robust education policy evaluation, as well as identifying key challenges. It takes a view of policy evaluation as an activity that takes place throughout the…

Previous research that explores the effect of ethnicity in the perception of speaker accentedness and personality traits often finds that Asian appearance contributes to a more accented and less competent impression. Importantly, most of the work done to date employed only Caucasian first language-speaking listeners; moreover, ethnicity and gender…

John Read is about to retire as Professor in Applied Language Studies at the University of Auckland. He previously taught applied linguistics, Teaching English to Speakers of Other Languages (TESOL) and English for Academic Purposes (EAP) at Victoria University of Wellington, the SEAMEO Regional Language Centre, the University of Texas El Paso,…

Reflection is an essential process for optimizing student learning outcomes in study abroad. Composing digital stories is a promising strategy for achieving high quality reflection. In this project we developed and explicated a rubric for assessing how students used the digital storytelling format to reflect on their study abroad experiences. We…

New Zealand has been leading the world in terms of the data handling, and in more recent years, data visualisation approach in its school statistics curriculum. In 2013, bootstrapping and randomisation were added to the senior secondary school (Ministry of Education 2012). This paper gives an historical perspective of the people and groups that…

We know little about what raters rely on when participating in accentedness perception tasks as their qualitative comments are rarely scrutinised. At the same time, we know that (assumed) social information influences listener behaviour. This study investigates rater attitudes to and stereotypes about speakers of different varieties of English,…

This essay engages questions of evaluator role and indigenous peoples participation in evaluation within colonial and decolonization contexts. Specifically, I critique the Western emphasis on cultural competence and contrast the utility of "mainstream" evaluation approaches alongside three indigenous inquiry models (Te Kotahitanga,…

Despite 11,000 years of honing evaluation skills in order to thrive in some of the harshest climatic conditions on the planet, there are few Alaska Native program evaluators and until a recent exchange with New Zealand "Maori", there was no collective vision for building Alaska Native capacity in program evaluation. This article tells…

This paper will highlight the successful evaluation approaches deployed for long-term exploration of the impact of a national policy initiative as well as some of the results and outcomes. An interactive feedback process informed decision-making at the national and local level thereby enhancing both the initiative and its implementation in…

The examination of doctoral theses controls an important academic threshold, yet practices are often private, codes non-specific, and individuals isolated. This article adds to recent investigation of the examination culture by reporting informal panel discussion amongst a total of 23 University of Auckland (New Zealand) faculty members as to…

This article documents the author's personal journey as a new evaluator traversing paradigms, continents, and timelines on a quest to discover how best to practice evaluation for the benefit of Maori people (indigenous to New Zealand). The journey has taken the author to a number of evaluation conferences in Australasia and North America, which…

Promotion of evaluation as a profession in Australia and New Zealand is discussed. The profession is challenged to make a commitment to utility, quality products and processes, and the development of skilled, trained professionals. Ways of overcoming barriers to the establishment of evaluation as a profession are outlined. (TJH)