Cause and Correlation: A Practitioner's Field Guide

Every decision-support system eventually gets asked a question it wasn't built to answer: not "what happened," not even "what's likely to happen," but "what happens if we change X." A predictive model trained on historical data can tell you that customers with more support tickets churn more often. It cannot, on its own, tell you whether reducing support tickets will reduce churn — because it was never trained to distinguish "tickets predict churn" from "tickets cause churn" from "some third factor causes both."

This distinction sounds academic until it costs a real budget. A team that reads a correlation as causation and invests in reducing ticket volume — without asking why tickets and churn move together — routinely discovers the intervention did nothing, because the actual driver was something upstream of both (a product issue, a pricing change, a bad onboarding cohort) that the ticket count was only ever a symptom of.

The three questions that separate correlation from a decision

Is there a plausible confounder? Before trusting a correlation, ask what third variable could produce both the input and the outcome without any causal link between them. Tickets and churn might both spike because a cohort of customers onboarded during a bad release — the tickets didn't cause the churn, the release did, and reducing tickets after the fact does nothing for churn from that cohort.

Does the timing support the causal direction you're assuming? A shocking amount of "X causes Y" reasoning falls apart the moment you check whether X actually precedes Y in the data, rather than being roughly simultaneous with it or — more commonly than teams expect — following it. Customers who are already planning to churn often generate more support tickets on their way out, not fewer, which reverses the causal story most dashboards imply.

Would a randomized comparison plausibly show the same effect? This is the practical test, not a purity requirement. If you can't imagine a version of this intervention that, run as an A/B test, would show the effect the correlation implies, you don't have a decision-ready finding — you have a hypothesis worth testing.

When you can't run the experiment

Randomized tests are the gold standard, but plenty of real decisions can't wait for one — you can't randomly assign customers to "worse product experience" to measure churn impact, and some interventions are one-shot (a pricing change, a policy rollout) where you only get one crack at it. This is where causal inference methods designed for observational data — difference-in-differences, instrumental variables, propensity matching, regression discontinuity — earn their keep. Each makes a different assumption about the world in exchange for a causal estimate without randomization, and the practitioner's job isn't to know all of them cold, it's to be honest about which assumption you're making and whether it's remotely plausible for your data.

Difference-in-differences, for instance, assumes that whatever's different between your treated and untreated groups would have trended the same way over time absent the intervention — the "parallel trends" assumption. It's testable (plot the pre-period trends and see if they actually run parallel) and it's violated more often than teams check for, especially when the treated group was selected because it looked different to begin with — which is exactly the selection bias the method was supposed to sidestep.

What we actually recommend building

Not a causal inference platform — for most organizations that's solving a problem they don't have yet. What earns its keep immediately is a lightweight discipline layered onto existing decision-support tooling: every recommendation surfaced to a decision-maker carries an explicit causal-confidence label — "experimentally validated," "causal estimate from observational data, assumption X," or "correlation only, not decision-ready" — so the person acting on it knows what kind of claim they're actually looking at. The technical methods matter less, in practice, than making sure the distinction is visible at the point of decision instead of buried in a methodology appendix nobody reads before approving the spend.