Universal screening has been proposed as a method for effectively and efficiently identifying students at risk for mental health concerns. As with traditional rating scales, screeners have historically been used in the absence of any user training. Recent research suggests, however, that systematic training can be used to enhance the psychometric defensibility of teacher rating data. Thus, the present investigation evaluated the effectiveness of a pilot teacher training intervention on improving identification of students with behavioral or emotional concerns. A total of 91 teachers (57 training, 34 control) participated in the study. Participants in the training group received information on the utility of universal screening and were provided with information regarding mental and behavioral risk in schools. Videos depicting examples of problem behaviors and student concerns were also provided, and trainees were given the opportunity to practice rating videos with performance feedback. Following training, teachers completed the Social, Academic, and Emotional Behavior Risk Screener (SAEBRS) for their 1,158 students in four urban elementary schools. Teachers also completed measures of perceived assessment usability and willingness and efficacy to engage in universal screening. Finally, schools provided extant data regarding each student's academic and behavioral functioning (e.g., benchmark assessments, suspensions, and disciplinary referrals). Structural equation modeling and moderation analyses indicated a stronger relationship between SAEBRS fall risk and end-of-year behavioral outcomes in the trained group, and this relationship was moderated by assessment acceptability. Implications for practice and future research are discussed.