An important element of good design, instructional or otherwise, is awareness of and being empathetic to the needs, wants, interests, values, and opinions of the intended audience. Typical approaches to learning about the subjective viewpoints of a particular audience have included survey instruments based on Likert-type items and open-ended…

Course evaluations can be used for curriculum improvement and have the potential to better the student learning experience. However, because most are based on Likert scales and open-ended feedback, understanding diversity in student opinion and uncovering optimal options for course change and improvement are often difficult. Alternatively,…

Whenever multiple observers provide ratings, even of the same performance, inter-rater variation is prevalent. The resulting "idiosyncratic rater variance" is considered to be unusable error of measurement in psychometric models and is a threat to the defensibility of our assessments. Prior studies of inter-rater variation in clinical…

This manuscript's purpose is to introduce Q as a methodology before providing clarification about the preferred factor analytical choices of centroid and theoretical (hand) rotation. Stephenson, the creator of Q, designated that only these choices allowed for scientific exploration of subjectivity while not violating assumptions associated with…

This article reports on an exploratory two-stage sequential mixed methods research study that investigated the views of university educators on the introduction of assessment methods other than essays, exams and dissertations within taught Masters programmes. In the first stage, interviews were conducted internationally with 45 participants and…