We evaluate two reforms that modified the procedures of recruitment and promotion in Italian academia to balance the preeminent role of the recruiting school and to counter nepotism. We theoretically derive the decision rule of the evaluation committees and test it against data including information from all selections to associate and full…

Literature revealed that the patterns/methods of scoring essay tests had been criticized for not being reliable and this unreliability is more likely to be more in internal examinations than in the external examinations. The purpose of this study is to find out the effects of analytical and holistic scoring patterns on scorer reliability in…

Introduction: This paper looks at the issue of inadequate and omitted citations in manuscripts by collecting the experiential opinions of researchers from the dual perspectives of manuscript reviewers and authors. Method: An online survey was conducted with participation from 207 respondents who had experience of reviewing and authoring research…

Generalizability theory (G theory) provides a broad conceptual framework for social sciences such as psychology and education, and a comprehensive construct for numerous measurement events by using analysis of variance, a strong statistical method. G theory, as an extension of both classical test theory and analysis of variance, is a model which…

The mini-clinical evaluation exercise (mini-CEX) is a well-established method of assessing trainees' clinical competence in the workplace. In order to improve the quality of clinical learning, factors that influence the provision of feedback are worthy of further investigation. A retrospective data analysis of documented feedback provided by…

There is a substantial literature on the importance of good-quality early childhood care and education services. There is also, however, some agreement that service quality is a nebulous concept which is difficult to define and therefore difficult to assess. While there is growing literature in the area, the views of one stakeholder group--that of…

Using generalizability (G-) theory and rater interviews as both quantitative and qualitative approaches, this study examined the impact of scoring methods (i.e., holistic versus analytic scoring) on the scoring variability and reliability of an EFL institutional writing assessment at a Turkish university. Ten raters were invited to rate 36…

Many medical schools have adopted multiple mini-interviews (MMI) as an advanced selection tool. MMIs are expensive and used to test only a few dozen candidates per day, making it infeasible to develop a different test version for each test administration. Therefore, some items are reused both within and across years. This study investigated the…

Testing strategies can either have a very positive or negative effect on the learning process. The aim of this study was to examine the degree of consistency in evaluating the practicality and logic of questions from a medical school pathophysiology test, between students and family medicine doctors. The study engaged 77 family medicine doctors…

The assessments designed for and analyzed in this study used a task-based language design template rooted in theories of language reflecting heteroglossic language practices and funds of knowledge learning theories, which were understood as transforming classroom teaching, learning, and assessment through continua of biliteracy lenses. Using a…

This article reports on a study of the quality assessment of doctoral dissertations, and asks whether examiner characteristics influence assessment of research quality in PhD dissertations. Utilising a multi-dimensional concept of quality of PhD dissertations, we look at differences in assessment of research quality, and particularly test whether…

The aim of the study was to investigate how raters come to their decisions when judging spoken vocabulary. Segmental rating was introduced to quantify raters' decision-making process. It is hoped that this simulated study brings fresh insight to future methodological considerations with spoken data. Twenty trainee raters assessed five Chinese…

This paper investigates the performance of holistic and analytic scoring rubrics in the context of EFL writing. Specifically, the paper compares EFL students' scores on a writing task using holistic and analytic scoring rubrics. The data for the study was collected from 30 participants attending an English undergraduate program in a Yemeni…

This article reports on a pedagogical project aimed at helping second language (L2) learners of English develop the ability to detect and appropriately interpret spoken sarcasm. The study used a pre- and posttest procedure to assess the development of learners' ability to both detect sarcasm and impute appropriate speaker intentions and attitudes…

In the present study we investigated the effect of test format on oral performance in terms of test scores and discourse features (accuracy, fluency, and complexity). Moreover, we explored how the scores obtained on different test formats relate to such features. To this end, 23 Iranian EFL learners participated in three test formats of monologue,…