Attribution Gaps: When Missing Outcomes Change the Winner

A clean dashboard can hide an unfinished argument

Campaign A costs less per outcome. The chart is green. Someone suggests moving budget. Then you ask what happened to the leads without an outcome, and the room gets quieter.

That is the point at which I want the calculation to slow down. A report can be arithmetically correct and commercially premature. It may count confirmed outcomes accurately while leaving enough unresolved records outside the denominator to change the conclusion.

An unresolved record is not a failed lead. It is a record whose result is not yet known within the definition and window you are using. Some will become losses. Some will turn out to have booked, bought or qualified elsewhere. Some will remain genuinely ambiguous. Giving all of them a value of zero is an assumption, even when the dashboard makes it look like a fact.

The practical question is narrower than whether your attribution is perfect: could the missing outcomes reverse the decision you are about to make? That question is testable before you commission another dashboard.

The original worked example: two campaigns, 96 possible resolutions

This is a synthetic example, not a client result or an industry benchmark. Both campaigns use the same currency, equivalent outcomes and equally mature observation windows. Each unresolved record is deduplicated, belongs to one campaign and can contribute at most one additional outcome.

Campaign A spent 4,000 and has 20 confirmed outcomes, with five records unresolved. Campaign B spent 5,000 and also has 20 confirmed outcomes, with 15 unresolved. The current cost per outcome is 200 for A and 250 for B. On confirmed results alone, A is cheaper.

If none of A's open records becomes an outcome, B needs six additional outcomes to become cheaper. Five creates a tie: 5,000 divided by 25 equals 200. Six takes B to 26 outcomes and a cost of 192.31. The apparently weaker campaign can overtake the leader without another dollar being spent.

But A also has unresolved records. We enumerated every integer resolution: zero through five additional A outcomes, crossed with zero through 15 additional B outcomes. That is 6 × 16, or 96 combinations. A is cheaper in 51, B in 43, and two are ties. Those counts describe the grid. They are not probabilities: we have no evidence that the combinations are equally likely.

The useful finding is that the ranking can reverse. We do not need to pretend the grid forecasts what will happen. We need to stop treating one unfinished slice of it as a settled result.

Use the missing-outcome stress test

The Attribution Stress Test runs this calculation with anonymous totals. It gives you the confirmed cost, the complete arithmetic range and the number of additional outcomes needed to overturn the current leader, assuming that leader gains none.

The lowest possible cost is spend divided by confirmed plus unresolved outcomes. The highest is spend divided by confirmed outcomes. These limits describe the possible resolutions of the records you supplied. They are not confidence intervals, causal estimates or predictions of future campaign performance.

1. Define the result. Pick one event, such as a held qualified meeting or a paid order. Write the inclusion and exclusion rules before reading the numbers.
2. Freeze comparable cohorts. Use the same currency, attribution rule and reporting window. Give both populations the same time to mature.
3. Separate confirmed and unresolved. Deduplicate within and across campaigns. A confirmed outcome cannot also sit in the unresolved count.
4. Calculate both bounds. Keep zero confirmed outcomes explicit. The current cost is not yet measurable; it is never a free outcome.
5. Test the decision. If one campaign stays cheaper even in its worst case against the other's best case, the ranking survives these missing records. Otherwise, repair the evidence first.
6. Name the next owner. Assign the unresolved cohort, a deadline and the source that will establish the result. Run the comparison again after readback.

Which missing records deserve attention first?

Start with the records that could materially affect the comparison. In the example, six recovered outcomes for B are enough to cross the first threshold. That makes B's unresolved cohort worth investigating. It does not justify cherry-picking six favourable records and ignoring the rest.

Use a fixed reconciliation rule for the whole cohort. Trace the original source to the person, the commercial object and the outcome. Keep exact matches, unmatched records and ambiguous matches separate. A familiar name is not a stable identity, and a calendar label is not proof of attendance.

The CRM handoff verification method lays out those joins. The work can reveal a source overwrite, a duplicate contact, an appointment in a different calendar or an outcome recorded outside the CRM. Each break needs a specific correction. A blanket attribution edit can make the chart look tidier while making the evidence worse.

The most useful readout says what changed: the fixed population, the number resolved, the remaining exceptions and the revised comparison. It does not turn a reconciliation into a claim that the campaign suddenly improved. The campaign may be unchanged. Your understanding improved.

A stable ranking still does not authorize a budget move

Suppose A stays cheaper across every possible resolution. That settles one narrow question about this cohort. It does not tell you what the next dollar will buy, whether A can scale, or whether its outcomes carry the same commercial value.

A low-cost appointment can miss the service area, fail qualification or never attend. A higher-cost opportunity can close faster at a higher margin. Keep booked, held, qualified, signed and paid events separate. The commercial handoff matters because every stage changes what the denominator means.

Unequal maturity is another trap. A campaign that started last week may have unresolved records because the sales cycle is still running. Calling that an attribution failure would be premature. Compare cohorts with equal time since acquisition, or explicitly model the unresolved period before drawing a conclusion.

This method also assumes each unresolved record can add no more than one equivalent outcome. It does not fit repeat orders, weighted pipeline, multi-touch fractional credit or cases where two campaigns claim the same customer without first reconciling identity. If the unit cannot be defended, neither can the range.

Publish the method, protect the client

You do not need to publish a customer's records to make a useful contribution. This article publishes a reproducible synthetic example, the formulas, the complete scenario count and the conditions under which the method fails. Anyone can challenge the arithmetic without learning which company supplied an operational problem.

Real client research needs a stricter publication boundary. Removing a company name is not always enough. A distinctive industry, exact event date and unusual transaction count can identify a business together. Aggregate only what the question requires, suppress small groups, and get permission for the exact claims that survive review.

Our Growth Constraint Benchmark is a separate, dated aggregate study with its own population and limitations. Its analytics key events are not interchangeable with CRM-confirmed leads. The lesson is to let each source say only what it can actually establish.

Try the Attribution Stress Test with a made-up scenario first. Then bring a comparable anonymous cohort. If a handful of unresolved outcomes can overturn the decision, that is your next piece of work.