Peer assessment training was motivated, developed and evaluated to address fairness in higher education group learning. Team-centric pedagogies, such as team-based learning have been shown to improve engagement and learning outcomes. For many instructors using teams, peer assessments are integral for monitoring team performance and ensuring…

Currently there are no assessment instruments available for upper-division thermal physics, though several introductory assessments are currently available. Notably missing from these introductory assessment are items targeting statistical mechanics. This leaves a gap in the content that can be assessed by upper-division thermal physics faculty.…

Student Experience surveys are used to measure student attitudes towards their campus as well as to initiate conversations for institutional change. Validity evidence to support the interpretations of these surveys' results, however, is lacking. The first purpose of this study was to compare three Differential Item Functioning (DIF) methods on…

Disability-as-diagnosis, disability-as-identity, and disability-as-experience as pressing measurement issues in learning outcomes assessment are described in this paper. Disability represents a critically important latent variable in many learning outcomes assessment plans unless it is a formalized part of analytic plans. Noting the potential for…

The purpose of this study was to investigate Chinese university professors' perceptions about the ethicality of classroom assessment practices. In a survey of Chinese professors, participants completed a questionnaire with 15 scenarios that depicted ethical and unethical assessment practices. Participants consisted of 555 professors from 143…

This paper introduces an empirical study testing three kinds of bias in higher education student assessment. All of them are connected to the repetitive use of the same test questions which may facilitate academic cheating. The "same tests effect" may appear if two or more groups of students are writing the same test one after the other…

The test-optional movement in the United States emerged largely in response to criticism of standardized admissions tests as inadequate and potentially biased measures of postsecondary promise. Although anecdotal reports suggest that test-optional policies have improved campus diversity, empirical research has not yet confirmed this claim.…

In this study the authors used differential item functioning analysis to examine if there were items in the Mental, Emotional, and Bodily Toughness Inventory functioning differently across gender and athletic membership. A total of 444 male (56.3%) and female (43.7%) participants (30.9% athletes and 69.1% non-athletes) responded to the Mental,…

This study examined the factorial and item-level invariance of Wong and Law's emotional intelligence scale (WLEIS) in a sample of 375 international students in U.S. universities. Confirmatory factor analysis (CFA) and differential item functioning (DIF) analysis were employed at the test and item level, respectively. International students from…

We focus on the identification of differential item functioning (DIF) when more than two groups of examinees are considered. We propose to consider items as elements of a multivariate space, where DIF items are outlying elements. Following this approach, the situation of multiple groups is a quite natural case. A robust statistics technique is…

Though ubiquitous, Likert scaling's traditional mode of analysis is often unable to uncover all of the valid information in a data set. Here, the authors discuss a solution to this problem based on methodology developed by quantum physicists: the state multipole method. The authors demonstrate the relative ease and value of this method by…

We present an extension of the logistic regression procedure to identify dichotomous differential item functioning (DIF) in the presence of more than two groups of respondents. Starting from the usual framework of a single focal group, we propose a general approach to estimate the item response functions in each group and to test for the presence…

We present a mixed-methods approach to community-based assessment design that engages tribal college and university faculty, students, and science educators, as well as experts in cultural knowledge from the Blackfeet and Diné (Navajo) nations. Information from cultural experts, gathered through a combination of sequential surveys and focus group…

Many measurement biases affect student evaluations of instruction (SEIs). However, two have been relatively understudied: halo effects and ceiling/floor effects. This study examined these effects in two ways. To examine the halo effect, using a videotaped lecture, we manipulated specific teacher behaviors to be "good" or "bad"…

In the current investigation, the authors examined the validity of the L-r and K-r scales on the recently developed Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF; Y. S. Ben-Porath & A. Tellegen, in press) in measuring underreported response bias. Three archival samples previously collected for examining MMPI-2…