Earlier this year, the message from tech companies to employees was clear: Use as much artificial intelligence in your work as possible.
Employees called it “tokenmaxxing,” with a token referring to a unit of AI use roughly equal to a word fragment. Employees at Meta and Amazon even competed on leaderboards that tracked token use.
Then came the bills from companies, like Anthropic and OpenAI, that provide AI tools — and they were not cheap. Now the tokenmaxxing era appears to be over.
Meta told employees last week that it would soon limit AI use after seeing an “exponential increase” in costs. In May, Uber said it had blown through its projected AI spending for the year in just four months, and it has placed some monthly limits on AI coding tools. Walmart also set limits for different AI tools. And Amazon and Meta have taken down the tokenmaxxing leaderboards.
In other words, “tokenminning,” short for “token minimizing,” is now in.
The reversal, within just a few months, underlines how AI use remains in flux as people try to figure out how to best use the tools.
“The biggest problem is this is all changing so fast, people and companies don’t know what to do,” said Rob May, the CEO of Neurometric, a startup that helps companies better use AI, and the author of “The Tokenminning Manifesto.”
“CEOs who did not know how to measure the AI savviness of their employees thought, ‘Well, who’s using the most tokens?’” he said, adding that the philosophy ended up promoting volume over efficiency.
OpenAI and Anthropic offer subscriptions that cost $10 to $200 a month for use of their AI models; when subscribers hit their usage limit, they are cut off. But the bulk of the revenue comes from offering tools to companies like Meta, Shopify and Amazon, which pay not only subscription fees but for the tokens used by their tens of thousands of workers. So the more tokens that are used, the more money the AI costs.
A simple task, like asking AI to summarize the transcript from a company meeting, may use a few hundred tokens. More complex requests, like writing code to build a new product or feature, can use tens of thousands.
The costs of using AI models have soared as they have become more powerful and consume more tokens. Anthropic’s newest AI model, Fable, is twice as expensive as its previous model, Opus. While there are cheaper models, many employees have fallen into the habit of using the most powerful models for everything, May said.
The ways that people use AI have also changed. Instead of just conversing with AI chatbots, engineers deploy AI “agents,” which can work on complex tasks for hours at a time. As a result, engineers can use tens of thousands of dollars’ worth of tokens each month.
Many companies said they were trying to be more strategic about AI spending after not seeing clear returns on their investment.
“If you’re not actually able to draw a direct line to how many useful features and functionality you’re shipping, that trade becomes harder to justify,” Andrew Macdonald, Uber’s chief operating officer, said in a recent podcast interview. “That link is not there yet.”
That’s not to say companies won’t keep spending big on AI. Meta told employees that it was on track to spend billions on AI use this year, but wanted to “find places we can spend less while getting similar or better business results.” Marc Benioff, the CEO of Salesforce, an enterprise software company, said his company planned to spend hundreds of millions on AI this year but now tracked “agentic work units” instead of tokens. The new metric is supposed to measure output, not just use.
Meta’s and Walmart’s limits on employee AI use were reported earlier by The Information and Bloomberg.
It’s unclear how “tokenminning” might affect the bottom lines of Anthropic and OpenAI. At the height of the tokenmaxxing era this year, the AI companies reported record revenues driven by the use of coding tools. Last week, Meta told its engineers to use its internal coding assistant, MetaCode, instead of third-party tools if possible.
Meta declined to comment, Anthropic did not provide a comment, and OpenAI did not respond to a request for comment. (The New York Times has sued OpenAI and Microsoft, claiming copyright infringement of news content related to AI systems. They have denied the suit’s claims.)
The clear path forward for companies, May said, is to use cutting-edge AI only on complex tasks that require it and substitute cheaper models for other instances.
Companies can save as much as 90% by opting for less advanced AI models, said Andy Markus, AT&T’s chief AI officer. He said his engineers were using the most powerful AI models for some tasks and the less powerful ones for most other actions.
“There’s an ebb and flow,” he said. “What we do find is that, for most use cases, the latest, greatest frontier model isn’t needed.”
While most of the respondents said that AI usage benefits themselves, 10 per cent of them responded that greater AI use benefits employers and clients. (Image: Unsplash) 
Share
