NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: ED659361
Record Type: Non-Journal
Publication Date: 2023-Sep-29
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Answering the Call: A Field Experiment to Improve Tutor Recruitment
Carly Robinson; Katharine Meyer; Xiaoyang Ye; Chastity Bailey-Fakhoury; Susanna Loeb
Society for Research on Educational Effectiveness
Motivation: The COVID-19 pandemic resulted in substantial disruption to K-12 schooling and students learning (Kuhfeld, Soland, & Lewis, 2021). Policymakers and educators have pointed to the potential of tutoring to support students' learning recovery, given that tutoring is one of the most effective educational strategies to help students attain educational mastery of challenging topics (Nickow et al., 2020; Robinson & Loeb, 2021). Despite interest and financial support to expand tutoring efforts, many localities have struggled to recruit enough tutors (White, Carey, O'Donnell, & Loeb, 2021). While historically the most effective tutoring programs have leveraged teachers or paraprofessionals as the tutors (Nickow et al., 2020), college students can also increase students' academic performance and often at a lower cost (Kraft et al, 2022). Yet it is an open question how best to recruit college students. Some work suggests emphasizing monetary gains may be more effective than messaging around intrinsic or career benefits (Bhanot & Heller, 2022; Ajzenman et al., 2021). In this field experiment, we test the effect of different recruitment strategies on the likelihood college students sign up to become tutors. Setting: We implemented the intervention at Grand Valley State University (GVSU), a public university in Michigan. In 2020, GVSU developed a tutoring service called "K-12 Connect" to connect college tutors with Michigan K-12 students. Prior to the start of the intervention, about 700 college tutors had held virtual tutoring sessions with 2,500 students across the state. Research Question and Intervention: We developed four email message variants to test against a control group email, examining whether (a) "any" motivational messaging increased tutor sign-up relative to a generic recruitment message (the pooled treatment effect) and (b) whether "specific" motivations for tutoring were more effective. The intervention included an initial email in June 2022 and a follow-up email a week later. Treatment emails varied in their subject lines, the email body description of the program benefits, and the application link text. The four treatments (and corresponding email subject lines) were (1) prosocial -- do you want to help Michigan children?, (2) monetary -- looking to earn money?, (3) career -- are you looking to build your resume?, and (4) social -- looking to meet other GVSU students? Our main outcomes of interest were (1) whether students opened the email, (2) whether students clicked through to the application, (3) whether students applied to become a tutor, and (4) whether students were hired as tutors. We preregistered exploratory heterogeneity analyses (e.g., by gender). Our field experiment was preregistered on the Registry of Efficacy and Effectiveness Studies (REES, #13360). Research Design: All students enrolled in the summer or fall 2022 semesters at GVSU in June of 2022 were randomization in the intervention. We emailed 15,860 undergraduates -- 7% freshmen, 24% sophomores, 26% juniors, and 43% seniors. About 38% of the sample was enrolled in the summer 2022 term. About 60% were female, 19% were non-white (GVSU was unable to provide more detailed race data), and about 9% were education majors. We blocked randomization on college level and summer 2022 term enrollment. To assess the impact of assignment to the treatment on the outcomes we consider, we use regression models of the following general form, which include limited baseline characteristics (primarily to explain residual variation in outcomes) and randomization blocks: Y[subscript ir] = [alpha] + [beta]T[subscript i] + [delta subscript r] + X[gamma] + [epsilon] [subscript ir]. Findings: Overall, 68.7% of students opened the email, 3.9% of students clicked on the application, 1.3% applied to become tutors, and 0.9% were hired. We did not find an overall pooled treatment effect of receiving "any" motivational message relative to the control email. Examining each treatment arm, we did not find significant differences in whether students opened the email. However, students who received the monetary motivational message were 2.8 percentage points more likely to click through to the application (a 7% increase off the control group). Students who received the career motivation were also 1.3 percentage points less likely to click through. Students in the monetary motivation condition were 1.7 percentage points more likely to apply to become tutors, a 196% increase in application rates relative to the control group. We estimate that if the monetary motivation were distributed to all 15,860, GVSU would have received 273 more applicants than if they sent everyone the control email. Interacting gender with treatment, we find female students in the monetary condition were 2.3 percentage points more likely to submit an application and 1.7 percentage points more likely to be hired, driving the overall monetary condition effects. Although descriptive, conditional on applying, the yield was significantly higher for students who received the pro-social messaging. About 45% of control group students who applied were hired, but 80% of prosocial treatment group students who applied were hired. This suggests that the types of students motivated by a prosocial framing may be either more attractive to the hiring committee or more likely to persist through the hiring process. We asked students on their application why they wanted to become a tutor -- despite the strong treatment effect of the monetary motivation, 50% of students said they wanted to work with children and 21% reported they wanted to gain valuable skills, with only 2% saying it was because tutoring paid well. We are in the process of extending the intervention with another cohort of GVSU students, testing whether combining the monetary motivation with other motivations (e.g., monetary + prosocial vs. monetary + career benefits) has differential effects on applications and hiring. We will have outcome data from that extension for the conference. Conclusion: Students report wanting to tutor to help the community, but recruitment efforts must also recognize that students face financial constraints and often need to pursue paid college work experiences. Our intervention suggests proactively assuring students that tutoring is a paid position significantly increases the likelihood students apply for tutoring positions. Our work also highlights common recruitment strategies that are not effective (e.g., highlighting the career or social benefits), offering strategies for other colleges and tutoring programs.
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Location: Michigan
Grant or Contract Numbers: N/A
Author Affiliations: N/A