THE APPARENT CONDITION

The data is good. The test was clean, the sample was sufficient, the results are legible. One option performs best, or several options perform within a narrow, defensible range of each other. In this moment, marketing organizations tend to treat the decision as effectively made — the numbers pointed somewhere, and pointing there is what remains to be done.

Sometimes the numbers really did decide it. Often they narrowed the field and left the actual decision still standing, unmade, dressed up as though the data had already resolved it.

THE MARKETING PROBLEM

Data can rule options out. It is much less reliable at ruling a single option definitively in, especially in decisions that involve more than performance — decisions that touch brand position, long-term consistency, reputational exposure, or how the organization wants to be understood over time. In those cases, multiple options can survive the data equally well, and the thing that separates them isn’t a number. It’s a judgment about which of the surviving options the organization actually wants to be.

The trouble isn’t that data is unreliable. It’s that “we have strong data” and “the data has determined this decision” get treated as the same statement, when the second is frequently false even when the first is entirely true. A test can show that Option A slightly outperforms Option B on the metric measured, while leaving completely unaddressed whether Option A is a position the organization can sustain, whether it fits the brand’s actual trajectory, or whether it creates risk the metric was never built to detect. The data answered its own question well. It didn’t answer the broader one leadership actually needed answered.

THE DISTINCTION

A data-informed decision is one where evidence has usefully narrowed the field of reasonable options, and a judgment is still required to choose among what remains.

A data-determined decision is one where the evidence itself, without further judgment, sufficiently establishes which option should be taken.

Genuinely data-determined decisions exist and are common for narrow, well-specified questions — which subject line performs better, which layout converts more, which send time gets more opens. These are legitimate cases where the data really has settled the matter. The failure occurs when this same posture — “the data decided it” — gets extended to decisions the data was never equipped to settle on its own: which brand position to commit to, which claim to make publicly, which risk is acceptable to take. These decisions can be informed by data. They cannot be fully determined by it, because they involve tradeoffs the metric doesn’t price in.

THE EVIDENCE

Performance data, by its nature, measures what it was built to measure and is silent on everything else. This is not a flaw in the data; it’s a boundary of it. Evidence on claims testing shows results can clearly indicate audience preference while providing no signal at all about whether the preferred option is consistent with the organization’s actual capability or position — a gap that only becomes visible when someone asks a question the test was never designed to answer.

This pattern recurs specifically around decisions with brand or positioning stakes. Positioning research has documented instances where the highest-testing option was not the option ultimately adopted, because further evaluation — evaluation the test itself did not perform — surfaced considerations the data had no mechanism for capturing: fit with existing market perception, credibility risk, long-term sustainability of the claim. In each of these cases, the data was accurate and the decision was still unmade until someone applied judgment the data could not supply.

There is also a temptation to describe a judgment-completed decision as though it were data-determined, because “the data showed us” can sound more objective than acknowledging that evidence narrowed the options and leadership still chose among them. This does not mean the judgment was wrong. It means the actual basis for the decision can become obscured, which makes it harder to revisit the decision honestly if circumstances change.

THE DECISION CONSEQUENCE

Once this distinction is visible, “we have the data” stops functioning as a complete justification for a marketing decision with brand, positioning, or reputational weight. The relevant question becomes narrower and more honest: has the data actually eliminated the alternatives, or has it simply narrowed them to a set that still requires a choice.

This changes how decisions get owned. If a decision is genuinely data-determined, the data can be cited as the reason. If a decision is data-informed but judgment-completed, the judgment needs to be named as what it is — a call made by someone, on grounds beyond the metric — rather than laundered through the data as though the numbers made the choice unassisted. This matters because a decision attributed to data that didn’t actually make it becomes very hard to reexamine later; nobody owns the reasoning, because officially, no reasoning beyond the data was involved.

THE IMPLICATION

Before treating a marketing decision as settled by evidence, leadership should be able to state plainly whether the data ruled out the alternatives or merely narrowed them. If more than one option remains reasonably supported once the data is accounted for, the decision is not finished — it has reached the point where someone has to choose, on grounds the data does not supply, and that choice deserves to be made visibly rather than attributed to a number that never made it.