USE CASE: How to Make Honest Budget Decisions When Marketing Attribution Is Broken

A real-world GenAI marketing use case: how a web design agency that steered budgets by attribution numbers it couldn’t trust used GenAI to triangulate imperfect signals and test causation with incrementality, choosing directional truth over false precision.

Marketing attribution is broken, and no dashboard admits it, which is why the precise-looking numbers agencies steer by are so often confidently wrong. This is what measuring honestly anyway looks like in practice: how a web design agency used GenAI to triangulate imperfect signals and lean on incrementality, trading a precise fiction for an honest approximation.

The Context: an Agency that Lived on Attribution

A web design agency whose whole promise is conversion, so measurement is the heart of the job. It ran on attribution data: which channel, which campaign, which touchpoint drove each conversion, and therefore where the next pound of budget should go. The numbers were precise, the dashboards were confident, and the decisions followed them.

The Challenge: Precise Numbers Nobody Could Trust

The trouble is that attribution was always broken, in ways the dashboard never admited. Last-click hands all the credit to the final touch and none to whatever created the demand; multi-touch models split credit by rules someone invented, not by what actually caused anything; and privacy changes (tracking opt-outs, cookie restrictions, consent gaps) mean a large and growing share of the journey is simply invisible, quietly filled in by the platforms with modelled estimates that can’t be audited. So the precise-looking numbers the agency steered by were a confident story built on correlation – which touchpoint was seen – dressed up as causation – which touchpoint worked. The real question, did this spend cause a result that wouldn’t have happened anyway, attribution can’t answer, because it observes journeys, it doesn’t test them. The agency wasn’t short of attribution data; it was short of attribution data it could trust. And a precise number you can’t trust is worse than an honest range, because you act on it, defunding a good campaign on an undercounted figure and never knowing.

A precise answer to the wrong question is still wrong: Attribution tells you which touchpoint was in the room when the sale happened, not which one caused it. It measures correlation and reports it as causation, to two decimal places, and that false precision is more dangerous than honest uncertainty, because it looks like truth. A precise answer to the wrong question is still wrong.

The GenAI Workflow: Triangulate, and Test Causation

The fix wasn’t a better attribution model – they’re all broken in their own way – it was to change what the agency trusted, and to point GenAI at honesty rather than false confidence. Three moves:

First, triangulation instead of a single source of truth: rather than believe one attribution model, the team used GenAI to combine several imperfect signals: the attribution models (plural, for their disagreement as much as their answers), self-reported “how did you hear about us” data, business-level efficiency (revenue against total spend), and the shape of results when spend changed, into one directional read, with the uncertainty kept visible rather than smoothed away.
Second, incrementality over attribution where it mattered: for the big budget calls, they ran holdouts, withhold the spend from a comparable group and see what actually changes, which tests causation instead of assuming it, and used GenAI to help design the tests and interpret the messy results.
Third, directional honesty: GenAI was instructed to report ranges and confidence, not a single clean number, and to flag where the data couldn’t support a conclusion. The agency stopped steering by a precise fiction and started steering by an honest approximation.

🖥️

The GenAI prompt:

You are a marketing expert helping a web design agency make sense of broken attribution – do NOT give me a single confident answer about what drove results. Here are the imperfect signals we have: [attribution models, self-reported source data, revenue vs total spend, results when spend changed, any holdout tests].

1. Triangulate them into a directional read of what’s likely working – where the signals agree, where they disagree, and roughly how much of the journey is simply invisible (tracking/privacy gaps).
2. Report ranges and confidence levels, not point estimates, and flag every place the data can’t actually support a conclusion.
3. Where a budget decision hinges on causation, tell me what incrementality test (holdout) would actually answer it, instead of guessing from attribution.

Do NOT smooth over the uncertainty or invent precision the data doesn’t have. Mark correlation-dressed-as-causation wherever you see it, and mark assumptions as CONFIRM WITH ME.

The caveat that decides whether this works: The temptation with broken attribution is to reach for a tool that will finally give you the clean answer, and GenAI, asked “which channel drove our sales?”, will happily provide one: confident, precise, and built on exactly the same broken data, which is in fact a trap. GenAI can’t fix attribution, and used carelessly it adds a layer of false precision on top of a false premise, laundering a guess into an authoritative-sounding number. It’s most useful here doing the opposite of what you’d instinctively ask for (surfacing uncertainty, showing where signals disagree, admitting what can’t be known), which takes deliberate instruction, because its instinct is to be helpfully conclusive.

Triangulation improves a directional read but doesn’t manufacture certainty: several broken signals agreeing is encouraging, not proof, and the only thing that actually tests causation is an incrementality experiment, which GenAI can help design but never replace. And the honest output (a range, a confidence level, an outright “we can’t tell”) is less satisfying than a clean pie chart, so the discipline to prefer it stays human. GenAI helps you see the uncertainty clearly; it can’t remove it, and shouldn’t pretend to.

The Result: an Honest Approximation Beats a Precise Fiction

The agency stopped trusting a number it couldn’t trust. Instead of steering by one attribution model’s precise-looking credit, it steered by a triangulated, directional read (several imperfect signals combined, with the disagreements and blind spots left visible) and, for the decisions that genuinely turned on causation, by incrementality tests that actually measured whether the spend changed anything. GenAI did the synthesis and helped design the tests, but it was pointed at honesty rather than false confidence: ranges, not point estimates; “we can’t tell” where that was the truth. The decisions got slower to feel certain and better grounded, because an honest approximation you understand beats a precise fiction you don’t. No invented figures here: the change is that the agency stopped mistaking attribution’s precision for accuracy, and started making budget calls on what it could actually defend.

Recommended KPIs to Follow

When attribution is broken, the metrics that matter are about the honesty of your measurement, not a tidier ROAS. Watch how you decide, not just what the dashboard reports. Here’s where the evidence sits and the direction this should push things. The point is the direction of travel, not a promised number.

Big Decisions Validated by Incrementality (not attribution alone)

The share of major budget calls backed by a holdout or lift test that actually measures causation, rather than an attribution model that only assumes it. It’s the clearest sign you’ve stopped mistaking correlation for cause.

Benchmark: Direction, not a promise: the industry consensus is that no single attribution source is trustworthy alone, and that incrementality (geographic holdouts, conversion-lift tests) is the “truth layer” that bypasses attribution debates by measuring what happens when you turn spend off (Stackmatix; Attriqs).

How much of your conversion data is actually observed versus modelled or missing, and whether the team knows the number. You can’t weight a signal honestly if you don’t know how much of it is a guess.

Benchmark: The gap is large and growing: attribution gaps have widened from ~30-40% of journeys (2021) to ~50-70% today, with ~75-85% of iOS users opting out of tracking, and platforms filling the void with unauditable modelled estimates (Ingest Labs; Adligator / Triple Whale).

Decision Confidence Calibration (fewer bad-data reversals)

Whether decisions are made with honest confidence (ranges, not point estimates), and whether that cuts the costly reversals that come from trusting a precise-but-wrong number. Fewer “we defunded the wrong campaign” moments is the payoff.

Benchmark: No public figure, an internal metric; the failure it prevents is well documented, though: acting on undercounted ROAS routinely defunds profitable campaigns that simply skew to untracked (often iOS) audiences (Ingest Labs).

Incrementality is the truth test, blind-spot awareness is the honesty, calibration is the payoff. The external figures are agency/vendor-sourced but broadly corroborated; treat them as direction. And note the through-line: the goal isn’t a more precise number, it’s a more honest one.

Why this Transfers

Every business steering by attribution is steering by a number that privacy changes have quietly broken, and the danger isn’t that it’s wrong, it’s that it’s wrong and precise. The transferable move is to stop chasing a perfect attribution model that no longer exists: triangulate several imperfect signals, test the big calls with incrementality, and insist that GenAI show you the uncertainty rather than hide it. An honest approximation you understand beats a precise fiction you don’t.