The View from Two Years Ago
Cast your mind back to late 2024. The world was in the throes of a generative AI gold rush. Companies across India and the globe were eagerly launching pilot programs, experimenting with everything from chatbots to code assistants. Corporate AI investment
was already significant, hitting $253 billion that year, but the mood was one of optimistic exploration. The costs were seen as manageable, largely confined to initial development and training for small, controlled user groups. Most organizations were focused on proving that AI could work, with the assumption that scaling up would be a straightforward next step. The primary costs were for building and training models, treated as a one-time capital expense. This was the era of the pilot project, where AI was a fascinating experiment rather than a core, and costly, operational pillar.
Deconstructing the '18-Fold' Shock
The headline-grabbing claim of an 18-fold cost increase from Gartner taps into a real and painful experience for many businesses. While a single, universal '18x' multiplier is hard to pin down, the sentiment is backed by extensive research. Gartner analysts have repeatedly warned of a 'rude awakening' when companies move from pilot to production, stating that a production-ready system can be 'orders of magnitude' more expensive. Research from McKinsey found that, on average, AI spending nearly quadruples when an application goes live. One case study cited a bank whose projected annual AI spend leaped from a manageable $3 million during trials to over $25 million once employees had ungoverned access to the tools. This explosion isn't an 18-fold increase in a single variable, but a compounding effect across infrastructure, usage, and complexity.
It's the Inference, Not Just the Training
The biggest shift in understanding AI costs between 2024 and 2026 has been the focus from training to inference. Two years ago, the conversation was dominated by the high cost of training massive models. Today, the real budget killer is inference—the cost of running the model to generate a response to a user's query. Unlike traditional software, which has a fixed cost, generative AI incurs a cost every single time it is used. Gartner projects that by 2028, the costs of model inference will make up at least 70% of a model's total lifetime cost. This turns what was once viewed as a one-off capital expenditure (training) into a massive, recurring, and often unpredictable operational expense that grows in lockstep with user adoption. The more successful your AI tool is, the more expensive it becomes to run.
The 80% You Don't See
The sticker price of a model's API call is deceptive. According to industry analyses, direct model and inference costs may only account for 15-20% of the total cost of ownership. The other 80% forms a 'hidden AI tax' buried in other parts of the business. This massive hidden cost includes the constant data engineering needed to feed the model; the ongoing maintenance to prevent model 'drift'; the human oversight required to govern outputs and manage risk; and the complex integration work needed to make the AI tool talk to other legacy systems. These are not one-time setup fees but continuous, labour-intensive processes that were drastically underestimated during the pilot phase, when such real-world complexities were often ignored.
The New Playbook for Indian Firms
For Indian businesses navigating this new, expensive AI landscape, the strategy must evolve. The question is no longer 'Can we build an AI tool?' but 'Can we afford to run it efficiently?'. The answer lies in shifting away from using massive, general-purpose models for every minor task—a practice some experts call the computational equivalent of 'taking a high-speed train to cross the street'. The new playbook prioritises architectural intelligence: using smaller, more efficient open-source models for routine tasks and reserving the expensive frontier models for only the most complex jobs. It also demands robust governance from day one, creating a centralised control plane to track usage, manage costs, and ensure every AI query is routed to the right model for the right price. The focus has decisively shifted from pure capability to proven, cost-effective ROI.
















