Scoring high-dimensional assessments (e.g., > 15 traits) can be a challenging task. This paper introduces the multilabel neural network (MNN) as a scoring method for high-dimensional assessments. Additionally, it demonstrates how MNN can score the same test responses to maximize different performance metrics, such as accuracy, recall, or…

As the field of higher education has used several methods to assess college teaching, the validation of these measures and how they relate to each other has been sparse. This study drew on data from a multi-institutional study of 587 college courses to examine the efficacy of two methods of measuring college teaching (class observation and…

The reliability and validity of course grades as measures of general education learning outcomes were assessed using the multitrait-multimethod (MTMM) proposed by Campbell and Fiske. Convergent validity and discriminant validity were assessed on college undergraduate students over a 12-year period. Two procedures were used to test the MTMM. The…

Practitioners and researchers interested in understanding student achievement, its predictors, and how it relates to other student outcomes are likely unaware of how the source information about achievement may offer subtly different pictures. This study applies multitrait-multimethod (MTMM) confirmatory factor analysis (CFA) within a structural…

The "Kobayashi Maru" is a training simulation that has its roots in the Star Trek series notable for its defining characteristic as a no-win scenario with no "correct" resolution and where the solution actually involves redefining the problem. Drawing upon these characteristics, we designed a board meeting simulation for an…

This paper examines how different test methods may tap different aspects of second language knowledge. It employs multiple-choice (MC) and constructed response (CR) items which yield distinct or convergent information in the computer delivered testing of English in its presentation of this factor. In order to examine the effects of test method, a…

Cultural differences between faculty and their students can create important challenges that affect the quality and efficacy of online teaching and learning. The objectives of this study were to: (a) create and pilot test an assessment for online faculty to measure culturally responsive teaching knowledge (CRT) and culturally responsive…

Estimation problems associated with the correlated-trait correlated-method (CTCM) parameterization of a multitrait-multimethod (MTMM) matrix are widely documented: the model often fails to converge; even when convergence is achieved, one or more of the parameter estimates are outside the admissible parameter space. In this study, the authors…

This article reports on a study of Australian teaching and learning centres to identify factors that contribute to their effective strategic leadership. These centres remain in a state of flux, with seemingly endless reconfiguration. The drivers for such change appear to lie in decision makers' search for their centres to add more strategic value…

We investigated the relationship between parent support and peer support as predictors of depression and self-esteem in college students. Several competing models of parental and peer influence were compared including a mediational model in which peer support was hypothesized to mediate the effects of parental support on adjustment. The results…

Three approaches of validation enquiry were applied on the data obtained from a proficiency test carried out with 3,398 PhD candidates as a partial requirement for entering PhD program in Iran. The data obtained from the reading section were subjected to an exploratory factor analysis (EFA). The EFA yielded nine factors in the reading section.…

This study assessed trait validity of the Reynolds Intellectual Assessment Scales' (RIAS) Verbal Index (VIX) and Nonverbal Index (NIX) scores in a group of college students. Using both observation of patterns and latent variable modeling of a multitrait-multimethod correlation/covariance matrix, the results indicate that the RIAS VIX scores…

Research on work as a calling is limited by measurement concerns. In response, the authors introduce the multidimensional Calling and Vocation Questionnaire (CVQ) and the Brief Calling scale (BCS), instruments assessing presence of, and search for, a calling. Study 1 describes CVQ development using exploratory and confirmatory factor analysis…

Ongoing professional development for college teachers has been much emphasized. However, previous research on learning environments has seldom addressed college students' perceptions of teachers' PCK. This study aimed to evaluate college students' perceptions of a physics teacher's PCK development using a newly developed instrument and workshop…

Introduction: The purpose of the study was to adduce evidence for estimating the construct validity of clinical competence measured through assessment instruments used for high-stakes examinations. Methods: Thirty-nine international physicians (mean age = 41 + 6.5 y) participated in high-stakes examination and 3-month supervised clinical practice…