This paper isolates and examines separately three distinct sources of error in essay scores: lack of agreement between raters; inconsistencies in performance within mode of discourse, and inconsistencies in performance between modes of discourse. Essay prompts in the Iowa Tests of Basic Skills (ITBS) Writing Supplement were designed to assess…

A large pediatric residency program conducted an extensive analysis of the reliability and validity of the rating forms used to evaluate the pediatric residents enrolled in the program. Data indicate that although the reliability of individual ratings is very low, several factors achieved acceptable levels of reliability when aggregated.…

The purpose of this study was to take an initial step toward developing sound and versatile predictive instruments (i.e., benchmark measures) in reading, writing, and math that can be used to assess high-school students' academic performance in a manner that will not only predict scores on high-stakes tests but will also be amenable to repeat…

Objective: The aim was to construct and test the reliability (utility, internal consistency, interrater agreement) and the validity (internal validity, concurrent validity) of a scale for home visiting social nurses to identify risks of physical abuse and neglect in mothers with a newborn child. Method: A 71-item scale was constructed based on a…

There is a critical need for research into the use of digital technologies to support the assessment of performance on complex tasks in schools. This paper reports on a component of a pilot study aimed at investigating the use of digital forms of performance assessment, manageable within schools, with high levels of reliability and capable of…

Treatment integrity (also referred to as "treatment fidelity," "intervention integrity," and "procedural reliability") is an important methodological concerning both research and practice because treatment integrity data are essential to making valid conclusions regarding treatment outcomes. Despite its relationship to validity, treatment…

Based in Shulman's idea of Pedagogical Content Knowledge, Technological Pedagogical Content Knowledge (TPACK) has emerged as a useful frame for describing and understanding the goals for technology use in preservice teacher education. This paper addresses the need for a survey instrument designed to assess TPACK for preservice teachers. The paper…

We examined the relationship between English as a Foreign Language (EFL) teachers' interpersonal behaviour and students' fluency in English in secondary education in China. A total of 160 students from four classes in the southwest part of China were asked to assess their teachers' interpersonal behaviour using the Questionnaire on Teacher…

This study assesses the psychometric properties of the Condom Barriers Scale (CBS), an instrument originally designed to measure women's perceptions and attitudes regarding male condom use, with a sample of men at high risk for human immunodeficiency virus (HIV). Participants include 590 male patients in drug abuse treatment involved in a…

Learning objects are interactive web-based tools that support the learning of specific concepts by enhancing, amplifying, and/or guiding the cognitive processes of learners. Research on the impact, effectiveness, and usefulness of learning objects is limited, partially because comprehensive, theoretically based, reliable, and valid evaluation…

The purpose of this study was to examine and compare the perceptions of undergraduate and graduate students of a selected state university in Texas attending the Multi Institutional Teaching Center (MITC)/The University Center (TUC) or the main campus regarding the effectiveness of student services. As universities face limited resources and…

Our recent review of content assessments revealed that language expectations and proficiencies are often implicitly embedded within the assessment criteria. Based on a review of performance assessments used in high school biology settings, we have found a recurring discrepancy between assessment scoring criteria and performance expectations. …

The Vanderbilt Assessment of Leadership in Education is a 360 assessment of principals' learning-centered leadership behaviors. The instrument was designed to provide formative and summative assessment to principals on the leadership behaviors most important to student learning. The purpose of this report is to describe the standard-setting…

This report is an analysis of the role of assessment portfolios in teacher learning. Over 18 months, 19 experienced science teachers worked in grade-level teams to design, implement, and evaluate assessments to track student learning throughout a curriculum unit, supported by semi-structured tasks and resources in assessment portfolios.…

The INTASC Principles, when used as the basis for developing appropriate measurement instruments to assess teacher dispositions, provide a viable approach to the diagnosis and remediation of skill-related affective performance in teacher candidates and also to meeting NCATE requirements for Standard 1. In this symposium, the development and use of…