This cross-sectional study explored students' conceptions of science learning via drawing analysis. A total of 906 Taiwanese students in 4th, 6th, 8th, 10th, and 12th grade were asked to use drawing to illustrate how they conceptualise science learning. Students' drawings were analysed using a coding checklist to determine the presence or absence…

In this paper, we briefly review the current literature on treatment integrity and discuss the relevance of this procedure for detecting, measuring and ensuring that the proposed mechanisms of change in cognitive behavior therapy, in this case of acceptance and commitment therapy (ACT; S.C. Hayes, Strosahl, & Wilson, 1999), take place. We discuss…

This article presents a study conducted by Cornelia Yarbrough and Jennifer Whitaker titled "Analysis of Reviewer Comments About Quantitative Manuscripts Accepted by the "Journal of Research in Music Education"." The study aims to analyze reviewers' comments for quantitative manuscripts with regard to the following categories: section discussed…

This study examines the usefulness of the Montreal Service Concept framework of service quality measurement, when it was used as a predefined set of codes in content analysis of patients' responses. As well, the study quantifies the interrater agreement of coded data. Two raters independently reviewed each of the responses from a mail survey of…

Functional behavioral assessments can consist of indirect, descriptive and experimental procedures, such as a functional analysis. Although the research contains numerous examples demonstrating the effectiveness of functional analysis procedures, experimental conditions are often difficult to implement in classroom settings and analog conditions…

In addition to validity and reliability evidence, other psychometric qualities of the PE Metrics assessments needed to be examined. This article describes how those critical psychometric issues were addressed during the PE Metrics assessment bank construction. Specifically, issues included (a) number of items or assessments needed, (b) training…

Behavioral Systems rely on valid measurement systems to manage processes and feedback and to deliver contingencies. An examination of measurement system components designed to track customer service quality of furniture delivery drivers revealed the measurement system failed to capture information it was designed to measure. A reason for this…

Science education reform has long focused on assessing student inquiry, and there has been progress in developing tools specifically with respect to experimentation and argumentation. We suggest the need for attention to another aspect of inquiry, namely "mechanistic reasoning." Scientific inquiry focuses largely on understanding causal…

This study examined the sexual messages depicted in music videos aired on MTV, MTV2, BET, and GAC from August 2, 2004 to August 15, 2004. One-hour segments of music videos were taped daily for two weeks. Depictions of sexual attire and sexual behavior were analyzed via a four-page coding sheet (interrater-reliability = 0.93). Results indicated…

Since the 1920s, textbook critics have maintained that textbooks should offer a homogenous editorial approach, including an acknowledgment of a mix of author opinion and scholarly research. Several researchers indicated that some textbooks are not homogenous. The purpose of this quantitative content analysis study was to examine whether…

The third edition of the "Boston Diagnostic Aphasia Examination" (Goodglass, Kaplan, and Barresi) introduced standardized procedures for coding discourse samples elicited using the well known Cookie Theft illustration. To evaluate the reliability of this discourse coding procedure, a transcribed sample was coded by 14 novice examiners…

School principals have complex jobs. To better understand the work lives of principals, this study uses observational time-use data for all high school principals in Miami-Dade County Public Schools. This paper examines the relationship between the time principals spent on different types of activities and school outcomes including student…

Current practices of reporting a single mean intercoder agreement in meta-analysis leads to systematic bias and overestimates reliability. An alternative is recommended in which average intercoder agreement statistics are calculated within clusters of coded variables. Two studies of intercoder agreement illustrate the model. (SLD)

This paper reports on a case study which identifies and illustrates sources of difference in agreement in relation to reliability in a context of quantitative content analysis of a transcript of an online asynchronous discussion (OAD). Transcripts of 10 students in a month-long online asynchronous discussion were coded by two coders using an…

Although protocol analysis can be an important tool for researchers to investigate the process of collaboration and communication, the use of this method of analysis can be time consuming. Hence, an automatic coding procedure for coding dialogue acts was developed. This procedure helps to determine the communicative function of messages in online…