This article reviews the evidence on the extent to which experts' perceptions of item difficulties, captured using comparative judgement, can predict empirical item difficulties. This evidence is drawn from existing published studies on this topic and also from statistical analysis of data held by Cambridge Assessment. Having reviewed the…

Although there is extensive evidence confirming the predictive validity of situational judgement tests (SJTs) in medical education, there remains a shortage of evidence for their predictive validity for performance of postgraduate trainees in their first role in clinical practice. Moreover, to date few researchers have empirically examined the…

Objectives: The purpose of this study was to examine the construct and predictive validity of the Adult-Adolescent Parenting Inventory (AAPI-2). Methods: The validity of the AAPI-2 was evaluated using multiple statistical methods, including exploratory factor analysis, confirmatory factor analysis, and latent class analysis. These analyses were…

The primary goal of this dissertation is to develop and provide preliminary validation for a new measure of culturally responsive practice. This instrument, which is called the Culturally Responsive Practice Scale (CRPS), includes items that reflect ways that teachers teach multicultural students in their classrooms. To accomplish the goal, three…

Wording effects associated with positively and negatively worded items have been found in many scales. Such effects may threaten construct validity and introduce systematic bias in the interpretation of results. A variety of models have been applied to address wording effects, such as the correlated uniqueness model and the correlated traits and…

Researchers examined the concurrent and predictive validity of a brief (12-item) teacher-rated school readiness screener, the Kindergarten Student Entrance Profile (KSEP), using receiver operating characteristic (ROC) curve analysis to examine associations between (N = 78) children's social-emotional (SE) and cognitive (COG) readiness with…

The purpose of this study is to translate and adapt the Sentiments Attitudes and Concerns about Inclusive Education Scale (SACIE) for use in the Turkish context. For this purpose, translated version of SACIE was administered to 304 and 368 preservice teachers (PTs) to perform exploratory and confirmatory factor analysis respectively. The result of…

Purpose: This study examines the psychometric properties of the Teamwork Scale for Youth, an assessment designed to measure youths' perceptions of their teamwork competency. Methods: The Teamwork Scale for Youth was administered to a sample of 460 youths. Confirmatory factor analyses examined the factor structure and measurement invariance of the…

Graduate admission has become a critical process in tertiary education, whereby selecting valid admissions instruments is key. This study assessed the validity of Graduate Record Examination (GRE) General Test scores for admission to Master's programmes at a technical university in Europe. We investigated the indicative value of GRE scores for the…

English language programmes provide established pathways for international students seeking university admission in countries such as Australia and the United Kingdom. In order to refer international applicants to appropriate levels and durations of English language support prior to matriculation into their main course of study, pathway providers…

Over 30 states have adopted the Early Childhood Environmental Rating Scale-Revised (ECERS-R) as a component of their program quality assessment systems, but the use of ECERS-R on such a large scale has raised important questions about implementation. One of the most pressing question centers upon decisions users must make between two scoring…

Recent methods have sought to improve precision in randomized controlled trials (RCTs) by utilizing data from large observational datasets for covariate adjustment. For example, consider an RCT aimed at evaluating a new algebra curriculum, in which a few dozen schools are randomly assigned to treatment (new curriculum) or control (standard…

In this article, I provide answers to the questions posed by Will Shadish about the history and development of the Schmidt-Hunter methods of meta-analysis. In the 1970s, I headed a research program on personnel selection at the US Office of Personnel Management (OPM). After our research showed that validity studies have low statistical power, OPM…

Two studies were conducted to examine and compare the construct validity of scores on the Junior Metacognition Awareness Inventory (JMAI) and problem-solving interview protocols. Participants consisted of 183 middle and high school students attending a university summer program for academically talented youth. Study 1 results indicated that JMAI…

The purpose of this paper is threefold: first, to introduce a conceptual model for assessing undergraduate students' integrated information and communication technology (ICT) literacy capacity that involves 12 items generated from the modified version of the Australasian Survey of Student Engagement (AUSSE) questionnaire (Coates, 2010); second, to…