Multilevel models are an increasingly popular method to analyze data that originate from a clustered or hierarchical structure. To effectively utilize multilevel models, one must have an adequately large number of clusters; otherwise, some model parameters will be estimated with bias. The goals for this paper are to (1) raise awareness of the…

Field experiments in education frequently assign entire groups such as schools to treatment or control conditions. These experiments incorporate sometimes a longitudinal component where for example students are followed over time to assess differences in the average rate of linear change, or rate of acceleration. In this study, we provide methods…

Multilevel modeling (MLM), also referred to as hierarchical linear modeling (HLM) or mixed models, provides a powerful analytical framework through which to study colleges and universities and their impact on students. Due to the natural hierarchical structure of data obtained from students or faculty in colleges and universities, MLM offers many…

Dimensionality is an important assumption in item response theory (IRT). Principal component analysis on standardized residuals has been used to check dimensionality, especially under the family of Rasch models. It has been suggested that an eigenvalue greater than 1.5 for the first eigenvalue signifies a violation of unidimensionality when there…

Despite the recent increase of structural equation modeling (SEM) in language testing and learning research and Kunnan's (1998) call for the proper use of SEM to produce useful findings, there seem to be no reviews about how SEM is applied in these areas or about the extent to which the current application accords with appropriate practices. To…

This study examined similarities and differences in how U.S. and Japanese middle-school science teachers teach science through inquiry. Classroom practices were examined through observations in the United States (N = 9) and Japan (N = 14). The observational data were coded and quantified based on the rubric that incorporated 2 dimensions: student…

Cluster sampling results in response variable variation both among respondents (i.e., within-cluster or Level 1) and among clusters (i.e., between-cluster or Level 2). Properly modeling within- and between-cluster variation could be of substantive interest in numerous settings, but applied researchers typically test only within-cluster (i.e.,…

Purpose: This purpose of this paper is to investigate the relationship between dimensions of commitment to the profession of business, and ascribed importance of various organisational characteristics to the first full-time job following graduation. Design/methodology/approach: Business administration students (n=446) completed surveys on…

This report examines theoretical and empirical issues related to the statistical power of impact estimates under clustered regression discontinuity (RD) designs. The theory is grounded in the causal inference and HLM modeling literature, and the empirical work focuses on commonly-used designs in education research to test intervention effects on…

This paper reports the reanalysis of data collected in a study of 3 determinants of classroom environment (viz. year level, subject, and school type) using multivariate analysis of variance and multilevel analysis. Data were collected from 2,211 students in Queensland Catholic and government schools. The Catholic School Classroom Environment…

This study evaluated the robustness of two recent methods for analyzing multivariate repeated measures when the assumptions of covariance homogeneity and multivariate normality are violated. Specifically, the authors' work compares the performance of the modified Brown-Forsythe (MBF) procedure and the mixed-model procedure adjusted by the…

Methods for the analysis of within-subjects effects in multivariate groups by trials repeated measures designs are considered in the presence of heteroscedasticity of the group variance-covariance matrices and multivariate nonnormality. Under a doubly multivariate model approach to hypothesis testing, within-subjects main and interaction effect…

Using the cluster generation procedure proposed by D. Steinley and R. Henson (2005), the author investigated the performance of K-means clustering under the following scenarios: (a) different probabilities of cluster overlap; (b) different types of cluster overlap; (c) varying samples sizes, clusters, and dimensions; (d) different multivariate…

Two Monte Carlo studies were conducted to examine the sensitivity of goodness of fit indexes to lack of measurement invariance at 3 commonly tested levels: factor loadings, intercepts, and residual variances. Standardized root mean square residual (SRMR) appears to be more sensitive to lack of invariance in factor loadings than in intercepts or…

An analysis of repeated measures designs is proposed that uses an empirical Bayes estimator of the covariance matrix. The proposed analysis behaves like a univariate analysis when sample size is small or sphericity nearly satisfied, but behaves like multivariate analysis when sample size is large or sphericity is strongly violated. (SLD)