Diagnostic classification models (DCMs) classify examinees based on the skills they have mastered given their test performance. This classification enables targeted feedback that can inform remedial instruction. Unfortunately, applications of DCMs have been criticized (e.g., no validity support). Generally, these evaluations have been brief and…

This article employs confirmatory factor analysis to evaluate the factorial validity and the cross-national comparability of the UTAUT constructs with respect to mobile learning in higher education in four Caribbean countries. Except for the measurement of one factor, the UTAUT constructs exhibit adequate reliability and validity. Though full…

The psychometric properties of the Revised Children's Manifest Anxiety Scale-Second Edition (RCMAS-2) were examined in a sample of 1,003 U.S. elementary and secondary students in Grades 2 to 12. Confirmatory factor analyses (CFAs) were performed comparing the five-factor (target) model consisting of three anxiety (Physiological Anxiety, Social…

As a result of this research, a quantitative model and a procedure have been developed to create an online mentoring effectiveness index (EI). To develop the model, mentoring and teaching effectiveness are defined, and then the constructs and factors of effectiveness are identified. The model's construction is based on the theory that…

Community engagement professionals and partners serve as, work with, study, and build the capacity of boundary spanners. To augment knowledge about these functions, the Weerts-Sandmann Boundary Spanning Conceptual Framework (2010) has been operationalized through a survey instrument to examine community engagement boundary-spanning behaviors by…

Significant momentum exists for next-generation assessments to increasingly utilize technology to develop and deliver performance-based assessments. Many traditional challenges with this assessment approach still apply, including psychometric concerns related to performance-based tasks (PBTs), which include low reliability, efficiency of…

The authors summarize evidence from a multiyear study with secondary students with reading difficulties on (a) the potential efficacy of primary-level (Tier 1), secondary-level (Tier 2), and tertiary-level (Tier 3) interventions in remediating reading difficulties with middle school students, (b) the likelihood of resolving reading disabilities…

The goal of this report was to test the use of sensor-based skill measures in evaluating performance differences in rifle marksmanship. Ten shots were collected from 30 novices and 9 experts. Three measures for breath control and one for trigger control were used to predict skill classification. The data were fitted with a logistic regression…

In this article we examine empirical evidence on the criterion, predictive, transfer, and fairness aspects of validity of a large-scale language arts performance assessment, referred to as the Performance Assignment (PA). We use multilevel models to avoid biased inferences that might result from the naturally nested data. Specifically, we examine…

Research was conducted to assess the applicability of the Instructional Systems Development (ISD) job analysis procedures to nine technical aviation maintenance military occupational specialties (MOS). Job analysis questionnaires were developed for each of the nine aviation maintenance MOS's. Research teams administered the questionnaires to…

Conducting a worthwhile assessment of the performance of senior leaders such as university presidents poses unique challenges for public institutions of higher education. One of the most difficult issues is determining the "content" and "format" of the assessment instrument. Due to the breadth and complexity of the job, the…

A proposed model for the validity of measures of professional competence treats validation as the evaluation of inferences drawn from test scores, focusing on evaluation, generalization, and extrapolation. The model is used to indicate strengths and weaknesses of assessments of professional competence: observations of performance, simulations, and…

Traditionally, studies investigating the relationship between mental abilities tests and performance on learning tasks have attempted to establish a direct relationship between these two domains by use of factor analytic techniques. It is suggested that an alternative approach can be formulated in terms of an information processing analysis of…

A synthetic performance test is a job performance test that has been degraded to some degree in the range of tasks covered or in the fidelity of stimulus/response features. Since further development is needed before synthetic performance testing is valid and efficient, this research project focused on three objectives: to (1) identify problems…

Some theoretical concerns for competency-based evaluation instruments are discussed, and means of examining these instruments for validity and reliability are presented. The areas of concern include descriptions of the behavior, the level of response, and the nature of the evaluation. Two different types of instruments are examined to determine…