The Student Evaluation of Teaching (SET) instrument provides insight for instructors and administrators alike, often touting high response-rates to endorse their validity and reliability. However, response-rate alone omits consideration for "adequate quantity of 'observational sampling opportunity' (OSO) data points" (e.g., high student…

Researchers and policymakers in the US and beyond increasingly seek to identify teaching qualities that are associated with academic achievement gains for K-12 students through effectiveness studies. Yet teaching quality varies with academic content and social contexts, involves multiple participants, and requires a range of skills, knowledge, and…

In this webinar, Dr. Drew Gitomer, professor at Rutgers University, shared results from recent studies of classroom observations that helped participants understand both general findings about the qualities of classroom interactions and also the challenges to carrying out valid and reliable observations. This Q&A addressed the questions…

There are situations within middle school settings where measurements of students and teachers are used for high-stakes decisions. For example, student performance is used as an indicator of teacher quality or determines student eligibility for particular types of support services. Given the high-stakes nature of these types of assessments,…

Student-teacher interactions are dynamic relationships that change and evolve over the course of a school year. Measuring classroom quality through observations that focus on these interactions presents challenges when observations are conducted throughout the school year. Variability in observed scores could reflect true changes in the quality of…

Passage of the Every Student Succeeds Act (ESSA) in December 2015 signaled a new era for teacher evaluation reform. Under ESSA, states and districts have greater autonomy to design and implement teacher evaluation systems independent of federal influence. This new flexibility brings with it new responsibilities and challenges for states and…

It has long been noted in research literature that there does not exist a shared definition of integration of science and mathematics. Classifying a particular set of integration practices by one of many labels has limited value. Moving past the definition debate, this study describes the development and testing of the Mathematics Integrated into…

Recent articles in "The Journal for Specialists in Group Work" have discussed credibility indicators for quantitative and qualitative studies (Asner-Self, 2009; Rubel & Villalba, 2009). This article extends upon these contributions by discussing measurement issues that are relevant to producers and consumers of quantitative group research. This…

During hearings in the late 1980s on education legislation, the frustrated chairman of the Education and Labor Committee of the U.S. House of Representatives, Rep. Gus Hawkins (D-CA), would sometimes lean over to his chief counsel and ask: "Why can't these people agree on the facts? If they did that, we could solve these problems quickly." That…

In this article, the authors clarify the role of the leadership team, providing a rationale for one integrated team to examine academic, social, and behavioral programming, with careful attention to including all key stakeholders. Next, the authors discuss the procedures for teaching all key stakeholders the comprehensive, integrated, three-tiered…

In this project, the researchers set out to develop a protocol to measure team behaviors in engineering education. The objective of this paper is the result of the attempt by the researchers to observe teams in the engineering classroom. The focus is the development of the protocol to measure team behavior and lessons learned from implementing…

The Autism Observation Scale for Infants (AOSI) was developed to detect and monitor early signs of autism as they emerge in high-risk infants (all with an older sibling with an autistic spectrum disorder). Here we describe the scale and its development, and provide preliminary data on its reliability. Inter-rater reliability both for total scores…

In the growing literature on the evaluation of program implementation, less has been said about evaluating program quality than about evaluating other aspects of program implementation. Furthermore, most articles and reports in the program-implementation evaluation literature have presented only brief descriptions of how implementation instruments…

This article suggests that quality assessment in the UK has been largely set apart from learning and teaching and reports on a pilot project at the Hull York Medical School which attempted to integrate students' evaluation of their clinical placements into the curriculum. It outlines the operational demands of this integrated method and compares…

This article examines issues in developing valid and reliable direct observation of behavior. Suggestions are made to minimize the problems that threaten validity and reliability. The discussion is concluded by an examination of costs and benefits of direct observation and who pays them and who benefits from them. (Contains 5 tables and 2 figures.)