Office hours are a part of nearly all science, technology, engineering, and mathematics (STEM) courses, with many academic and affective benefits for students who attend. However, despite their ubiquity, there has only been limited past work examining the sources of knowledge students draw from to shape their perceptions of office hours, as well as the factors that influence students' experiences in STEM course office hours. Here, we conducted semistructured interviews with 20 students enrolled in an introductory STEM course to investigate what motivates students to attend STEM course office hours, the factors that shape their experiences in office hours, and what influenced their conceptions of office hours. We situate our work using expectancy value theory and racialized opportunity cost, identifying that students' experiences with office hours in high school and how an instructor describes office hours in a class and on the syllabus influence students' expectancies and perceived value and costs for attending office hours. In addition, we find that most students are motivated by the need for help with course content, though some students report a higher perceived cost of attending office hours when they have lower self-efficacy in the course. We conclude by providing recommendations for instructors to encourage student engagement with office hours.