In previous messages, I’ve communicated the vision, strategy, and operations of FirstGen Ahead. This month my message focuses on our impact measures. The measures extend beyond the pilot phase and include outcome measures as well as input/process measures. The metrics are at an individual level (for students and coaches) and at an organizational level. Data are collected through monthly coach and student reports, surveys, online coach and student sessions, and student assignments.
For student outcomes, we’re interested in both tangible results (e.g., acquired a summer internship) and self-perception (e.g., career self-efficacy) over the course of the program. For the tangible results, we’re interested in understanding whether the students’ social networks have grown over the course of the program based on an expansion of their network map. Students drew their original network map in October. Bottom-line tangible outcome measures include acquiring a summer internship (paid and in career field of interest), and obtaining a strong, full-time first job post-college (where a college degree is required, benefits are offered, and there are opportunities for growth). For the perceptual measures, we have baseline data on all students from the beginning of the program, and we will re-administer the career self-efficacy survey in December, 2020. The final perceptual outcome measure for students is satisfaction with participating in FirstGen Ahead. We have collected information on the perceived value of the first three months of the program (as reported in the last newsletter) and students will be asked to complete additional assessments in May and December. As of now, one student has committed to a summer internship and another student has an eight-week semester internship.
Outcome measures for coaches are: 1) interest in participating as a coach after the pilot phase has completed; 2) satisfaction with their involvement in the program; and 3) recruiting other coaches for future cohorts of students. Data on these measures will be collected in December, 2020.
At an organizational level after the pilot phase, the key outcome measure is a sustainable/stable funding base and a scalable or replicable model.
We are also tracking input/process measures. One of these measures for both students and coaches is participation rates. For example, as of February 1st, the rate of students participating in the monthly online sessions is 63% and the participation rate of coaches on the monthly coach support calls is 76%. For students, we are also tracking follow-through on assignments, agreements reached with coaches, and ratio of applications submitted to job offer rate for both internships and first position post college.
At an organizational level, the input/process measures that are being tracked is conversion rate (attempted/realized) for coach volunteers, students, organizational partnerships, and grant proposals. At this stage, the conversion rates are 58% (coach volunteers), 72% (students), and 100% (organizational partnerships).
Understanding our impact is critical for our mission, strategy and operations going forward and necessary to secure future funding.
Susan Gershenfeld, PhD
