Rethinking AI in the Organisational Chart

At my startup, one thing became clear very quickly. AI did not belong to any single team. The technology team not only maintained the underlying tools and infrastructure but also used AI extensively to accelerate development, improve code quality, and increase engineering efficiency. At the same time, its impact extended across the organisation. In product and game design, AI helped us move from ideas to solutions faster. In art, it enabled rapid prototyping of concepts. In marketing, it accelerated the creation of campaigns, creatives, and content.
What stood out to me was not just the presence of AI, but how naturally it was embedded into everyday work across teams. You could not point to a box on the organisational chart and say, “AI sits here.” It was everywhere, embedded in how work got done.
This, however, is still the exception. In most organisations, the question of where AI belongs remains unresolved. Is it an IT capability, a strategic function, or something owned by business teams? The ambiguity is not trivial. It is one of the key reasons why, despite heavy investment, most AI initiatives fail to move beyond pilot stages into meaningful business impact. Only a small fraction of organisations have successfully scaled AI into sustained, value-generating workflows. In trying to solve this, organisations tend to default to one of three approaches.

Some organisations place AI within IT as this ensures governance, reliability, and security. But it also constrains AI to a support role. IT is optimised for stability, not experimentation. As a result, most efforts remain limited to task-level automation instead of driving real workflow transformation.
Others place AI within strategy or innovation teams. This creates visibility and ambition, but not execution. Without integration into day-to-day operations, AI remains stuck in pilots. This is where most organisations fail, caught between activity and outcomes.
A third approach embeds AI within business units. This enables speed and relevance. However, without coordination, it leads to duplication, fragmented systems, and governance risks. Many firms are already seeing the rise of “shadow AI,” where adoption outpaces control.
Each model solves for one problem, but creates another.

The organisations that are moving ahead are not choosing between these models. They are combining them. The emerging answer is a federated structure. A central team builds and governs the core capabilities, such as data, infrastructure, and access and then the business teams apply AI within their workflows, where context and value exist. This balance allows organisations to scale without losing speed.
This model works because AI creates value not through isolated tools, but by reshaping workflows end to end. Organisations that use AI as an add-on see only marginal gains, but those that embed it into core processes see measurable improvements in speed, efficiency, and decision-making.

Looking back, this is exactly what I experienced at my startup. The tech team enabled AI and improved its own productivity through it, but did not control its usage. Each function adapted AI to its own needs, whether it was improving design decisions or scaling content production. The result was not just efficiency, but faster iteration and better decision-making across the board.
A similar principle is visible at scale. Walmart deliberately positioned its AI leadership closer to the business rather than treating it as a back-end IT function. By aligning AI initiatives with workforce and operational priorities, the company was able to deploy its “My Assistant” tool to over 50,000 corporate employees within just 60 days. This was not just a technology rollout, but an organisational shift. By ensuring AI was owned as a business capability rather than a technical project, Walmart was able to move rapidly from experimentation to enterprise-wide adoption.

The same shift is underway in India. Enterprises are investing aggressively and are more optimistic than global peers, yet only a small fraction have embedded AI into core workflows. Although some companies are beginning to close this gap. Reliance Industries is embedding AI across telecom, retail, and digital services to drive personalisation and operational efficiency, as noted by AInvest. Tata Consultancy Services and Infosys are integrating AI into enterprise solutions at scale, as reported by The Times of India. Flipkart is using AI across recommendations, forecasting, and supply chains, embedding it directly into revenue-generating workflows, according to Renaissance Advisors.
What differentiates these organisations is not access to AI, but integration. They are not asking where AI should sit. They are redesigning how work happens.
The answer, then, is clear. AI should not sit in a single function. It should be centrally enabled and widely applied across the organisation.

Abhiyash is a PGDGM student at XLRI Jamshedpur.