What the end of tokenmaxxing means for AI ROI | Daily Reports Online
From businesses exhausting yearly AI budgets in just months to some imposing limits on staff AI use, it’s clear heavy token consumption or ‘tokenmaxxing’, is reaching its limits.
Instead of incentivizing and measuring business output, many are measuring consumption and usage.
While cutting AI usage seems like the natural solution, this doesn’t always work in practice. In fact, this can result in genuinely useful projects being pulled.
Latest Videos FromTechRadar
Without a reliable way to measure AI ROI, companies cut against the only metric they can see: consumption.
EMEA Field CTO at Apptio, an IBM Company.
We know a lack of performance benchmarks and traceability is translating to poor ROI with Gartner estimating that 84% of finance leaders have not been able to measure the ROI of AI initiatives.
To give enterprises the confidence to navigate this next phase, business leaders need to prioritize getting a clear picture of AI spend from beginning to end.
How We Got Here
The rapid adoption of generative AI has introduced a new consumption model and traditional IT financial management needs to adapt to keep up. For a long time, organizations optimized their budgets around cloud and on premises workloads.
However, these new tools function in a different way. Costs vary based on the complexity and accuracy of a prompt or even the type of model being used. In other words, the inherent variability of LLMs has made accurate cost tracking more difficult.
The problem is only being made more complex by the introduction of AI agents which can increase expenses because of unpredictable token consumption, heavy GPU usage and fast scaling. Unlike standard AI chatbots, these are not static tools.
These agents work using continuous, background loops independent of human operators which can generate multiple queries to solve difficult tasks. Because this reasoning loop happens autonomously, it can make it trickier to understand how much it’s costing to run an agent.
The Blind Spot
Another hurdle is visibility. Because of the rapid adoption of AI tools, spend is rarely centralized, distributed across a complex mix of business units, infrastructure, vendor APIs and engineering teams. Enterprise cloud and API bills are also unlikely to be updated, or interpreted in real-time meaning it gets even harder to understand what has been spent.
As a result, this is forcing a shift in how organizations measure success and spend. It is not enough to track the raw, isolated figure of cost per token. To make sure that enterprises have a clear understanding of what they are paying for and what they are getting in return, it’s important that team leaders have the frameworks in place to keep a firmer hold on budgets.
A business’s financial practices must evolve at the same pace as its technology adoption.
Value Over Volume
So, what replaces the trial-and-error approach that has defined AI adoption so far? Having spent years working with businesses, first through the cloud transition and now enterprise AI, I’ve seen that sustainable returns depend on rethinking how we define and measure productivity in the first place.
Many businesses have been encouraging workers to use AI wherever possible, but few have implemented specific AI metrics that can tie together, higher usage to improved outcomes.
For example, has the process of taking a product from concept to production sped up or become less expensive? Just measuring intermediate steps like code check-ins, ticket closures and cases doesn’t necessarily align with business value.
To do this, a benchmark must be established. Businesses need to know what a process, from manpower to tools already costs them without AI so they can make the right call. Without that baseline, any gain is guesswork.
This is where frameworks like Technology Business Management and FinOps earn their place. Both practices are aimed at making sure that every aspect of spend is understood and tied to a key business objective. In my experience they also help build a culture that instils accountability amongst teams when it comes to their role in managing IT spend.
There is a need to dismantle the silos that can keep costs out of sight and instead focus on treating technology spend as a real-time product variable.
Sustained Financial Intelligence
To make true progress and create value with AI, teams must see it as a measurable business driver. This transition requires leaders not to see success as how many times an employee has turned to AI or logged into the latest tool, or just having an AI tool as a part of a business process. Real ROI comes when enterprises can connect money spent directly to improved outcomes, that affect the business output.
Put simply, it’s about getting a clear and honest picture of spending so smarter choices can be made and investment in the right areas can be prioritized. The result? A more accurate understanding of costs and what AI projects are pushing the business forward as opposed to just being vanity projects.
We’ve listed the best business laptop deals.
This article was produced as part of TechRadar Pro Perspectives, our channel to feature the best and brightest minds in the technology industry today.
The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here:
