The use of inter-rater reliability (IRR) methods may provide an opportunity to improve the transparency and consistency of qualitative case study data analysis in terms of the rigor of how codes and constructs have been developed from the raw data. Few articles on qualitative research methods in the literature conduct IRR assessments or neglect to…

Item characteristics can have a significant effect on survey data quality and may be associated with measurement error. Literature on data quality and measurement error is often inconclusive. This could be because item characteristics used for detecting measurement error are not coded unambiguously. In our study, we use a systematic coding…

Purpose of the Study: Missing data are a common problem encountered while implementing measurement instruments. Yet the extent to which reliability, validity, average discrimination and difficulty of the test results are affected by the missing data has not been studied much. Since it is inevitable that missing data have an impact on the…

Although, collectively, results of earlier direct observation studies suggest momentary time sampling (MTS) may offer certain technical advantages over whole-interval (WIR) and partial-interval (PIR) recording, no study has compared these methods for measuring engagement in young children in naturalistic environments. This study compared direct…

Units of measure are critical in many scientific fields. While instructors often note that students struggle with units, little research has been conducted about the nature and extent of these difficulties or why they exist. We investigated calculus students' unit use in area and volume computations. Seventy-three percent of students gave…

Considering scoring validity as encompassing both reliable rating scale use and valid descriptor interpretation, this study reports on the validation of a CEFR-based scale that was co-constructed and used by novice raters. The research questions this paper wishes to answer are (a) whether it is possible to construct a CEFR-based rating scale with…

This manual provides guidance and documentation for users of the longitudinal kindergarten-fourth grade (K-4) data file of the Early Childhood Longitudinal Study, Kindergarten Class of 2010-11 (ECLS-K:2011). It mainly provides information specific to the fourth-grade round of data collection. The first chapter provides an overview of the…

Reliability generalization (RG) is a meta-analysis that combines and synthesizes reliability coefficients from different studies to ascertain the average observed reliability across studies. An RG study was conducted on previously reported data from 16 samples of the Overexcitability Questionnaire--Two (OEQII) with a combined "N" of 5,275.…

This revised data file includes corrections that were provided to NCES as a result of a special collection effort designed to address data quality issues found in the 1a release of this file. In May 2012, NCES became aware of data errors for key data items for several schools on the published version of the SY 2009-10 school file; in some cases…

The comparative format used in ranking and paired comparisons tasks can significantly reduce the impact of uniform response biases typically associated with rating scales. Thurstone's (1927, 1931) model provides a powerful framework for modeling comparative data such as paired comparisons and rankings. Although Thurstonian models are generally…

The impacts of incorrect responses to reverse-coded survey items were examined in this simulation study by reversing responses to traditional Likert-format items from 700 administrators in randomly selected schools in a 7-county region in central Arkansas that were obtained from an archival dataset. Specifically, the number of reverse-coded items…

The authors report a meta-analysis of individual differences in detecting deception, confining attention to occasions when people judge strangers' veracity in real-time with no special aids. The authors have developed a statistical technique to correct nominal individual differences for differences introduced by random measurement error. Although…

This manual provides guidance and documentation for users of the longitudinal kindergarten-fourth grade (K-4) public-use data file of the Early Childhood Longitudinal Study, Kindergarten Class of 2010-11 (ECLS-K:2011), which includes the first release of the public version of the third-grade data. This manual mainly provides information specific…

In light of a recent revelation that Gersten (1985) included erroneous information on one of two programs for English Language Learners (ELLs), the authors re-calculate results of their earlier meta-analysis of program effectiveness studies for ELLs in which Gersten's studies had behaved as outliers (Rolstad, Mahoney & Glass, 2005). The correction…

School principals have complex jobs. To better understand the work lives of principals, this study uses observational time-use data for all high school principals in Miami-Dade County Public Schools. This paper examines the relationship between the time principals spent on different types of activities and school outcomes including student…