Research Findings: Given the variable nature of early childhood settings, practitioners and researchers need better guidance on what conditions influence observations conducted within early childhood settings (National Research Council, 2008). Using 230 observations from 23 three- and four-year-old children, we conducted a Generalizability study…

Purpose: Estimating the sequential associations between educators' and children's talk during language learning interactions requires careful consideration of factors that may impact measurement stability and resultant inferences. This research note will describe a preliminary study that used generalizability theory to understand the contribution…

Both school and district administrators use the results of standardized, large-scale tests to inform decisions about the need for, or success of, educational programs and interventions. However, test results at the school level are subject to random fluctuations due to changes in cohort, test items, and other factors outside of the school's…

We evaluated the score stability of the Framework for Teaching (FFT), a prominent observation instrument used for teacher evaluation. Three raters each scored 200 reading and mathematics lessons taught by 20 kindergarten teachers. Using Generalizability theory analyses, we decomposed the FFT's Classroom Environment, Instruction, and Total scores…

Integrating basic principles, values, and practices of education for sustainability in early childhood education (ECE) is important for developing and developed countries. In this context, there is a need to identify and evaluate indicators that measure sustainable practices in ECE programs. This study was conducted in 18 preschools in Turkey to…

Running Records provide a standardized method for recording and assessing students' oral reading behaviors and are excellent formative assessment tools to guide instructional decision-making. This study expands on prior Running Record reliability work by evaluating the extent to which external raters and teachers consistently assessed students'…

Research Findings: We evaluated the score stability of the Mathematical Quality of Instruction (MQI), an observational measure of mathematics instruction. Three raters each scored, independently, 100 video-recorded lessons taught by 20 kindergarten teachers in the spring. Using generalizability theory analyses, we decomposed the MQI's score…

Accurate and rapid identification of students displaying behavioral problems requires instrumentation that is user friendly and reliable. The purpose of the study was to evaluate a multi-item direct behavior rating scale called the Direct Behavior Rating-Multiple Item Scale (DBR-MIS) for disruptive behavior to determine the number of…

The Learning Target Rating Scale (LTRS) is a measure designed to evaluate the quality of teacher-developed learning targets for embedded instruction for early learning. In the present study, we examined the measurement dependability of LTRS scores by conducting a generalizability study (G-study). We used a partially nested, three-facet model to…

We examined how raters and tasks influence measurement error in writing evaluation and how many raters and tasks are needed to reach a desirable level of 0.90 and 0.80 reliabilities for children in Grades 3 and 4. A total of 211 children (102 boys) were administered three tasks in narrative and expository genres, respectively, and their written…

We examined how raters and tasks influence measurement error in writing evaluation and how many raters and tasks are needed to reach a desirable level of 0.90 and 0.80 reliabilities for children in Grades 3 and 4. A total of 211 children (102 boys) were administered three tasks in narrative and expository genres, respectively, and their written…

Adapted from the Early Childhood Environment Rating Scale-Revised, the Chinese Early Childhood Program Rating Scale (CECPRS) is a culturally comparable measure for assessing the quality of early childhood education and care programs in the Chinese cultural/social contexts. In this study, 176 kindergarten classrooms were rated with CECPRS on eight…

Indicators of academic achievement for bilingual students can be inaccurate due to linguistic heterogeneity. For indigenous populations, language shift (the gradual replacement of one language by another) is a factor that can increase this heterogeneity and poses an additional challenge for valid testing. We investigated whether and how indigenous…

Language in the Individuals With Disabilities Education Improvement Act (IDEIA) allows the use of response-to-intervention (RTI) methodology in the identification of specific learning disabilities. However, there is no consensus on decision rules using curriculum-based measurement of oral reading fluency (CBM-R) for defining responsiveness. The…

While there are appropriate tools to progress monitor academics, there is no universally accepted tool to progress monitor social behavior. The current study proposed the development of a series of brief behavior rating scales to correspond to important social skills domains on the Social Skills Rating Scale (Gresham&Elliott, 2008) and to…