The Data-Driven DELUSION

The audit was supposed to be routine. The team had prepared. Documents had been created, reviewed and stored across the right systems. On paper, everything existed. But when the auditor asked for specific control evidence, no one could find it. Folders were opened, links were checked, and people started messaging each other in quiet panic. The document was there. It always had been. But in that moment, it might as well not have existed.

The organisation had data. It did not have understanding.

That distinction, unremarkable as it sounds, is at the centre of one of the most consequential and least examined problems in how modern businesses are led.

The belief driving most data investment is simple: more information leads to better decisions. Dashboards have grown richer, reports deeper and metrics more granular. By every visible measure, organisations are more informed than ever. But the quality of decisions has not kept pace. In many leadership environments, it has quietly regressed. The problem is not a shortage of data. It is that data abundance has produced a style of decision-making that optimises for safety over substance, and justification over judgement.

The Defensibility Trap

The shift is behavioural more than technical. In organisations where every decision requires a documented rationale, leaders learn quickly that being defensible matters more than being right. A failed call backed by solid data is forgivable. A successful one made on instinct remains suspect. So the incentive quietly inverts. People stop asking what the best decision is and start asking which decision they can best defend. Experience is discounted because it doesn’t fit on a slide deck. Pattern recognition built over years gets overridden by a metric that tells a cleaner story. The organisation does not announce this shift. It does not need to. People figure it out on their own, and they adapt accordingly.

Underlying this is a more fundamental issue. Data is always a representation of reality, not reality itself. It is filtered, delayed and shaped by whatever someone decided was worth measuring. A green dashboard does not mean the business is healthy. It means the business is performing well on what is being tracked, which is very different.

Nokia’s internal metrics in 2007 showed strong market share, loyal customers and healthy margins. They did not show that consumers were about to want something Nokia had not imagined building. Kodak had equally clear visibility into its film business right up until that visibility became irrelevant.

In both cases, the data was accurate. It was also dangerously incomplete. The organisations were not failed by bad information. They failed because they assumed their information was sufficient.

What the Numbers Never Show

The signals that go unmeasured are often the most important ones. Consider a retail chain where store managers had been flagging the same customer complaint informally for 18 months. It never made it into a report because there was no field in the system to capture it. By the time the issue appeared in structured data, it had already cost the company a meaningful chunk of repeat business.

This is not an unusual story. Customer hesitation during a sales conversation, the silence after a strategy is announced at a town hall, a pattern of small grievances that each seems too minor to escalate individually. These carry real information. But because they are not structured, they are not treated as data and are filtered out long before they reach the people who most need to hear them.

Judgment Beyond the Dashboard

Data also has a structural ceiling that no investment can raise. It is rooted in what has already happened. It can surface patterns and reduce known uncertainty, but it cannot tell you what is about to change or whether a market is ready for something that does not yet exist. When Reed Hastings committed Netflix to streaming in 2007, broadband penetration data was ambiguous, and the licensed content library was thin. No model would have returned a confident green light. The decision required judgment operating beyond what the numbers could confirm. That is precisely the capacity that defensive, data-dependent cultures erode over time. Organisations that only move when the data is conclusive will consistently arrive late because, by the time the data is conclusive, the window has usually narrowed.

The goal was never to collect more data. It was to decide better. Those two things are not the same, and conflating them has quietly cost organisations more than they realise. The leaders who navigate complexity well are not the ones with the most information. They are the ones who know which information to trust, which to question, and when the most important input in the room is not on any screen. That is not a rejection of data. It is a more honest understanding of what data is actually for.