Many methods are used to measure interrater reliability for studies where each target receives ratings by a different set of judges. The purpose of this study is to explore the use of hierarchical modeling for estimating interrater reliability using the intraclass correlation coefficient. This study provides a description of how the ICC can be…

Researchers in the field of early adolescence interested in quantifying the environmental influences on a response variable of interest over time would use cluster sampling (i.e., obtaining repeated measures from students nested within classrooms and/or schools) to obtain the needed sample size. The resulting longitudinal data would be nested at…

Randomized experiments are generally considered to provide the strongest basis for causal inferences about cause and effect. Consequently randomized field trials have been increasingly used to evaluate the effects of education interventions, products, and services. Populations of interest in education are often hierarchically structured (such as…

Numerous studies have suggested that academic stress has negative impact on adolescents' psychological function, few of those studies, however, considered whether and how the impact of stress on adolescents' emotional states is moderated by corresponding regulation. This study aimed to examine the fluctuation of emotional states before and after…

While most prior research has focused on the frequency of family meals the issue of which elements of family mealtime are most salient for adolescents' well-being has remained overlooked. The current study used the experience sampling method, a unique form of time diary, and survey data drawn from the 500 Family Study (N = 237 adolescents with…

Multilevel modeling techniques are becoming more popular in handling data with multilevel structure in educational and behavioral research. Recently, researchers have paid more attention to cross-classified data structure that naturally arises in educational settings. However, unlike traditional single-level research, methodological studies about…

Mind wandering--mentation unrelated to one's current activity and surroundings--is a ubiquitous phenomenon, but seemingly competing ideas have been proposed regarding its relation to executive cognitive processes. The control-failure hypothesis postulates that executive processes prevent mind wandering, whereas the global availability hypothesis…

Educational researchers commonly use the rule of thumb of "design effect smaller than 2" as the justification of not accounting for the multilevel or clustered structure in their data. The rule, however, has not yet been systematically studied in previous research. In the present study, we generated data from three different models…

Intraclass correlations are used to summarize the variance decomposition in popula- tions with multilevel hierarchical structure. There has recently been considerable interest in estimating intraclass correlations from surveys or designed experiments to provide design parameters for planning future large-scale randomized experiments. The large…

Using data collected through classroom videotaping, student surveys, and the Experience Sampling Method (ESM), the present study explored associations between teacher-provided intellectual challenge, two types of support (instrumental and emotional), and students' momentary academic engagement in high school science classrooms. Results of 3-level…