We proposed an extended form of the Govindarajulu and Barnett margin of error (MOE) equation and used it with an analysis of variance experimental design to examine the effects of aggregating student evaluations of teaching (SET) ratings on the MOE statistic. The interpretative validity of SET ratings can be questioned when the number of students…

This study uses decision tree analysis to determine the most important variables that predict high overall teaching and course scores on a student evaluation of teaching (SET) instrument at a large public research university in the United States. Decision tree analysis is a more robust and intuitive approach for analysing and interpreting SET…

The student evaluation of teaching process is generally thought to produce reliable results. The consistency is found within class and instructor averages, while a considerable amount of inconsistency exists with individual student responses. This paper reviews these issues along with a detailed examination of common measures of reliability that…

Effective teamwork skills are essential for success in an increasingly team-based workplace. However, research suggests that there is often confusion concerning how teamwork is measured and assessed, making it difficult to develop these skills in undergraduate curricula. The goal of the present study was to develop a sustainable tool for assessing…

To refine selective admission models, we investigate which measure of prior achievement has the best predictive validity for academic success in university. We compare the predictive validity of three core high school subjects to the predictive validity of high school grade point average (GPA) for academic achievement in a liberal arts university…

There is a lively discussion on how to evaluate competence-based higher education in both evaluation and competence research. The instruments used are often limited to course evaluation or specific competences, taking a rather narrow perspective. Furthermore, the instruments often comprise predetermined competences that cannot be adapted to higher…

Rubrics have attained considerable importance in the authentic and sustainable assessment paradigm; nevertheless, few studies have examined their contribution to validity, especially outside the domain of educational studies. This empirical study used a quantitative approach to analyse the validity of a rubrics-based performance assessment. Raters…

Prediction accuracy of academic achievement for admission purposes requires adequate "sensitivity" and "specificity" of admission tools, yet the available information on the validity and predictive power of admission tools is largely based on studies using correlational and regression statistics. The goal of this study was to…

Peer review feedback, developed to assist students with increasing the quality of group reports and developing peer review skills, was added to a master's level Climate Change Policy and Planning unit. A pre- and post-survey was conducted to determine whether students found the process a valuable learning opportunity: 87% of students responding to…

The psychometric measures of accuracy, reliability and validity of peer assessment are critical qualities for its use as a supplement to instructor grading. In this study, we seek to determine which factors related to peer review are the most influential on these psychometric measures, with a primary focus on the accuracy of peer assessment or how…

We present an equation, derived from standard statistical theory, that can be used to estimate sampling margin of error for student evaluations of teaching (SETs). We use the equation to examine the effect of sample size, response rates and sample variability on the estimated sampling margin of error, and present results in four tables that allow…

Student engagement has become a key feature of UK higher education, but until recently there has been a lack of data to track, benchmark and drive enhancement. In 2015 the first full administration ran in the UK a range of survey items drawn from the US-based National Survey of Student Engagement (NSSE). This is the latest example of international…

Viewing student engagement as a multidimensional construct, this study explored the motivation and engagement of undergraduate students in China. A sample of 1131 students from 10 full-time universities in Beijing participated in a survey. The results showed that the Motivation and Engagement Scale for university/college students is a promising…

The aim of this study was to assess the effect of the introduction of peer review processes on the quality of multiple-choice examinations in the first three years of an Australian medical course. The impact of the peer review process and overall quality assurance (QA) processes were evaluated by comparing the examination data generated in earlier…

Discusses statistical procedures for increasing test unreliability due to guessing in multiple choice and true/false tests. Proposes two new measures of test unreliability: one concerned with resolution of defined levels of knowledge and the other with the probability of examinees being incorrectly ranked. Both models are based on the binomial…