ERIC Number: ED659402
Record Type: Non-Journal
Publication Date: 2023-Sep-30
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Evaluation of Accelerate, Complete, Engage (ACE) at the City University of New York: The ASAP Model for Bachelor's Degree-Seeking Students
Jing Zhu; Diana Strumbos; Michael Scuello
Society for Research on Educational Effectiveness
Background/Context: Accelerated Study in Associate Programs (ASAP) is a comprehensive program developed by the City University of New York (CUNY) that has demonstrated remarkable success for associate degree-seeking students, nearly doubling the three-year graduation rate in a randomized controlled trial (Scrivener, et al., 2015). The model has been replicated in seven states, with similar impacts (Miller, et al., 2020), and has been noted as an evidence-based model by researchers, policymakers and funders. Many public institutions serving bachelor's degree-seeking students also struggle with low on-time graduation rates -- nationally, the most recent four-year graduation rate for first-time full-time students was 42.4% and rates were lower for students who were Black (24.4%) and Hispanic (32.6%) than White (46.9%) and Asian or Pacific Islander (50.6%) (USDOE, 2021). At CUNY, the most recent four-year graduation rate was 36.2%, with similar differences by race and ethnicity (CUNY OAREDA, 2023). To address these low graduation rates, CUNY implemented a version of the ASAP model, called Accelerate, Complete, Engage (ACE), for bachelor's degree-seeking students. This study seeks to test the effectiveness of ACE to understand whether the ASAP|ACE model works for bachelor's degree students as well. Research Questions: (1) ("Confirmatory") Do students who participate in the CUNY ACE program have a higher "on-time" (four-year) bachelor's degree graduation rate than they would have had if they did not participate in the program? and (2) ("Exploratory") For all outcomes, do the results vary for students across different subgroups of sex, race/ethnicity, and socioeconomic background? Four-year graduation is defined as having earned a bachelor's degree prior to the beginning of fall 2022 (yes vs. no). ACE is hypothesized to increase "on-time" graduation rates and help students graduate more quickly. Five-year graduation will also be analyzed for the final report. Setting: The study was launched in fall 2018 at John Jay College of Criminal Justice, the first CUNY senior college to implement the ACE model. Participants: ACE is designed to serve a broad population of first-time students. Students were eligible for the study if they met criteria including: applying for financial aid, meeting residency requirements and enrolling in approved majors. A total of 570 students enrolled in the study, with 285 assigned to each condition. Close to half of participants identified only as Hispanic or Latino/a (48.3%), with smaller percentages identifying only as White (14.2%), Black (10.2%) and Asian or Pacific Islander (9.7%). The majority of participants were Female (70.0%) and were eligible for Pell grants (71.8%). Students were on average 18 years old at the start and half (50.4%) were the first generation in their family to go to college. Program Model: Students were initially eligible to receive support for up to four years, although funding was later secured for a fifth year. ACE is a comprehensive program that includes the following components: (1) Early Engagement and Community Building: An ACE orientation in the summer prior to matriculation and community building activities offered each semester; (2) Academic Momentum: Expectations included attempting at least 15 credits each semester and enrolling in summer or winter sessions as needed to stay on track; (3) Advising and Career Development: Advisors had relatively small caseloads (maximum of 150 students) and were assigned to each student, staying with them from entry through graduation, which allowed them to build strong relationships and provide personalized support. Students were required to meet their advisor one-on-one each month and attend a monthly group seminar. Data and technology tools were used to help advisors effectively serve students. Students received career development support through workshops and activities; (4) Academic Support: Students were strongly encouraged to utilize tutoring or other services; and (5) Financial Supports: Tuition and fee gap coverage, funds for textbooks and course materials, and a MetroCard for public transportation on New York City subways and buses. Research Design: The study used an experimental design in which students were randomly assigned to the program or control condition with a 50:50 ratio. Baseline equivalence analysis confirmed that the two groups were equivalent across all key variables. Data Collection and Analysis: Data were merged from the CUNY Institutional Research Database and the National Student Clearinghouse (NSC), capturing data on enrollment and degrees at postsecondary institutions nationwide. Intention-to-treat (ITT) analyses were conducted using logistic regressions with pre-program control variables and effect sizes were calculated to measure the practical importance of findings in addition to statistical significance. Findings: The ACE program had a statistically significant positive impact on the likelihood of graduating with a bachelor's degree within four years. ACE students had a higher graduation rate (58.8%) than the control students (46.4%), a difference of 12.4 percentage points with a notable effect size, considered substantively important (Cox index = 0.304). Exploratory analyses found positive effects for nearly all subgroups, although only those for the largest subgroups were statistically significant. Effects for Hispanic or Latino/a, Black, White and Asian or Pacific Islander students were close in size (ranging from 16.7 to 20.3 percentage points) and effects were also similar in size for male and female students (13.0 and 11.4 percentage points). The program had a larger effect on students who were not eligible for Pell grants (21.3 percentage points) than those who were eligible (11.1 percentage points). Conclusions: This study found that ACE has a statistically significant and substantively important positive impact on four-year graduation, with similar estimated effects across lines of race/ethnicity and sex. CUNY has expanded ACE with a focus on serving transfer students with an associate degree, providing a future opportunity to analyze impacts across colleges and to better understand the generalizability of the model. Study limitations include: the sample size was not large enough to rigorously investigate potential differential effects and estimate whether ACE narrows equity gaps; implementation data were not collected to assess service contrast; the study relied only on administrative data and did not include qualitative data or reflect student voices. The results from the study to date suggest that the ACE program has provided critical student support and that it may be a promising model for institutions to adopt as they invest in student success initiatives.
Descriptors: Undergraduate Students, Acceleration (Education), Financial Aid Applicants, Academic Advising, Student Personnel Services, Academic Support Services, Career Counseling, Transitional Programs, Program Evaluation
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Location: New York (New York)
Grant or Contract Numbers: N/A
Author Affiliations: N/A