This research provides a direct comparison of effect size estimates based on structural equation modeling (SEM), item response theory (IRT), and raw scores. Differences between the SEM, IRT, and raw score approaches are examined under a variety of data conditions (IRT models underlying the data, test lengths, magnitude of group differences, and relative size of reference and focal groups). Results show that all studied methods perform similarly. All methods tend to underestimate effects as effect sizes become larger. SEM-based approaches to effect size estimation perform somewhat better at shorter test lengths, whereas approaches based on IRT and raw score perform somewhat better at longer test lengths. Although these differences between methods are detectable, they are small in magnitude. (Contains 4 tables and 2 figures.)