Tracking Outcomes into Year Five

A provost we worked with put the problem better than we could: "We know exactly what happens to a student for four years, and then they walk across the stage and become a rumor." Every institution has strong data on enrolled students — attendance, grades, financial aid, advising touchpoints, all captured continuously because the systems of record are built around active enrollment. The moment a student graduates, every one of those systems stops collecting, because none of them was ever designed to track a person who is no longer a student. What's left is whatever the alumni office can gather through newsletters, reunion RSVPs, and the occasional LinkedIn search — which is not a measurement system, it's a fan club with a spreadsheet attached.

This matters more than it looks like it should, because the outcome that actually justifies a degree program — did this credential change this person's employment and earnings trajectory — doesn't resolve in year one after graduation. Entry-level placement is noisy: it reflects the hiring market that particular spring as much as it reflects program quality. The signal that actually distinguishes a strong program from a weak one, in our experience running this analysis across several institutions, stabilizes somewhere between years three and five, once graduates have had time to move past their first job, and the trajectory — not just the starting point — becomes visible.

Why the data decays faster than people expect

Three mechanisms compound against you, and they compound faster than most institutional research offices plan for:

Contact information has a half-life of roughly eighteen months. Email addresses issued by the institution get deactivated. Personal emails change. Phone numbers turn over. Without an active re-verification process, a clean alumni contact list from graduation is down to maybe 60 percent deliverable within two years and considerably worse by year five — not because alumni are hiding, but because nobody's contact information stays static for five years unless something is actively maintaining it.

Self-selection bias gets worse every year, not better. The alumni who respond to a five-year survey are disproportionately the ones doing well — settled career, comfortable sharing an update, still emotionally connected to the institution. Someone struggling, or who's had a rough few years, is systematically less likely to respond, which means a naive survey-response outcome measure doesn't just have missing data, it has missing data that's correlated with the outcome you're trying to measure. A 35 percent response rate skewed toward success stories will show you a program performing better than it actually is, with no flag in the data telling you that's what happened.

Attribution gets genuinely harder to defend, not just harder to collect. Even with perfect contact data, a five-year-out earnings number reflects the degree, the labor market the graduate entered, whatever the graduate did with their career since, and factors with no relationship to the institution at all — a spouse's job that required relocation, a health event, a market downturn in their specific sector. Institutions that report five-year outcomes as if they were a clean measure of program value are usually overclaiming what the number can support, and a board or accreditor who pushes on the methodology will find the attribution chain thinner than the number suggested.

What we build instead of a five-year survey

A single wave of retrospective survey outreach at the five-year mark is the default approach and it's the weakest version of this — low response rate, worst self-selection bias, no way to see the trajectory that got a graduate from their first job to wherever they are now. What holds up better is a maintained, longitudinal instrument built on three design choices:

Verify contact information annually, not at the five-year mark. A short, low-friction annual touchpoint — even something as light as a one-question "still at this employer" prompt tied to a benefit alumni actually want, like a transcript request portal or a professional-network mailing — keeps contact data from decaying in the first place. It's cheaper to maintain a contact list continuously than to reconstruct one from scratch five years out, and the continuous version doesn't carry the same recency-skewed self-selection problem a single retrospective survey does.

Supplement self-reported data with licensed, structured labor-market data where the institution has legal grounds to use it. Employment-verification and wage-record matches — through state longitudinal data systems where institutions have access, or National Student Clearinghouse-style aggregators for continued-education tracking — fill in outcomes for alumni who never respond to a survey at all, which is the group a survey-only approach systematically misses. This doesn't eliminate attribution problems, but it removes the response-bias problem entirely for the alumni it covers.

Report trajectories, not point estimates, and report the coverage rate alongside the outcome. A single number — "68 percent employed in field five years out" — invites exactly the overclaiming problem above. A trajectory view (starting role, role at year three, role at year five, for the cohort you actually have data on) paired with an honest statement of what share of the graduating class that cohort represents lets an institution say something true and defensible, instead of something clean and unsupportable. Boards and accreditors respond better to a program that can show its work than to one over-claiming precision it doesn't have — a 55 percent-covered trajectory with visible methodology has held up to accreditor scrutiny in ways a 90 percent-covered single-number outcome, built on an undisclosed survey response rate, has not.

None of this collapses the fundamental difficulty of tracking people who've left your systems. It does convert "we don't really know" into a defensible, if partial, answer — which, five years out, is the realistic bar, not the mirage of a clean number nobody's data can actually support.