Many standardized educational tests include groups of items based on a common stimulus, known as "testlets". Standard unidimensional item response theory (IRT) models are commonly used to model examinees' responses to testlet items. However, it is known that local dependence among testlet items can lead to biased item parameter estimates…

This study compares the dependability of four classroom assessment scoring methods. Generalizability theory (G) and alternative decision (D) are used to measure the results of students' classroom assessment scores and compare the results of the four scoring methods on variability of rater by person variance and the level of G and D coefficients…

The Vanderbilt Assessment of Leadership in Education (VAL-ED) is a multirater assessment of principals' learning-centered leadership. The instrument was developed based on the Standards for Educational and Psychological Testing. In this article, we report on the validity and reliability evidence for the VAL-ED accumulated in a national field…

The Autism Diagnostic Interview-Revised is one of the "gold standard" diagnostic tools for autism spectrum disorders. It is traditionally administered face-to-face. Cost and geographical concerns constrain the employment of the ADI-R for large-scale research projects. The telephone interview is a reasonable alternative, but has not yet been…

In this article we provide an overview of the nature and scope of multisource feedback (MSF) and provide empirical evidence of its reliability, validity, and feasibility in one of the health professions. The overall internal consistency reliability (Cronbach alpha) of MSF instruments is generally greater than 0.96 for self and informants such as…

Because it is often impractical or impossible to obtain school transcripts or records on subjects, many researchers rely on college students to accurately self-report their academic record as part of their data collection procedures. The purpose of this study is to investigate the validity and reliability of student self-reported academic…

The aim of this study was to report the preliminary validation results for the Pain Rehabilitation Expectations Scale (PRES). The PRES is a clinical tool developed to measure the expectations about rehabilitation treatment and outcome for people with back pain. Fifty people with chronic back pain were recruited from 11 physiotherapy outpatient…

Over the past fifteen years many schools have utilized aggression prevention programs. Despite these apparent advances, many programs are not examined systematically to determine the areas in which they are most effective. One reason for this is that many programs, especially those in urban under-resourced areas, do not utilize outcome measures…

Primarily used in corporate and organizational contexts, this study evaluates the psychometric properties of the 30-item "Organizational Listening Survey" ("OLS") as a measure of listening behavior with a sample of undergraduate college students. The first study analyzed 1,475 students' self-reports of their listening behavior on campus,…

The main purpose of this article is to demonstrate how halo effects may be detected and quantified using two independent ratings of the same person. A practical illustration is given to show how halo effects can be avoided. (Contains 2 tables, 7 figures, and 2 notes.)

Developing normed treatment outcome measures is important to research addressing treatment effectiveness and to improved clinical care. The Preschool Outcome Questionnaire (POQ) is a new measure designed for use with preschool children aged two to six. Designed in collaboration with parents and clinicians, the POQ is brief, easy to administer,…

Youth with learning disabilities (LD) are at an increased risk for anxiety disorders and valid measures of anxiety are necessary for assessing this population. We investigated the psychometric properties of the Multidimensional Anxiety Scale for Children (MASC; March in Multidimensional anxiety scale for children. Multi-Health Systems, North…

Reliability can be estimated using structural equation modeling (SEM). Two potential problems with this approach are that estimates may be unstable with small sample sizes and biased with misspecified models. A Monte Carlo study was conducted to investigate the quality of SEM estimates of reliability by themselves and relative to coefficient…

This study provides preliminary analysis of reliability and validity of scores on the Self-Efficacy Teacher Report Scale, which was designed to assess teacher perceptions of self-efficacy of students aged 8 to 17 years. (Contains 3 tables.)

There is a growing recognition of the value of measuring fidelity of implementation (FOI) as a necessary part of evaluating interventions. However, evaluators do not have a shared conceptual understanding of what FOI is and how to measure it. Thus, the creation of FOI measures is typically a secondary focus and based on specific contexts and…