SUMMARYCorporate enthusiasm for “tokenmaxxing” is fading as companies find that heavy AI use can raise costs without clear productivity gains. Firms advised by Bain & Company are increasingly adopting model-routing systems so simpler tasks go to cheaper AI models and complex work goes to more capable ones. Lower-cost open-source models from Chinese startups such as Moonshot and Zhipu are also pressuring prices across the AI market.
An anonymous reader quotes a report from the Associated Press: A corporate fad of "tokenmaxxing" on artificial intelligence technology is hitting its limits as workplaces throwing AI at everything are seeing the costs rise without a similar spike in productivity. What started as tech industry-fueled springtime hype over squeezing as much AI-generated work as possible out of products like OpenAI's ChatGPT and Anthropic's Claude has shifted to a summertime backlash. [...] Just a few months ago, Silicon Valley executives were promoting high token consumption as a signal of high-performing employees. The stereotypical tokenmaxxer was staying up late -- perhaps ignoring their significant other -- while orchestrating an army of 24-hour AI agents performing work on their behalf. [...] The trend boosted revenue for leading AI large language model developers like Anthropic and OpenAI, but it fizzled as it became apparent it wasn't necessarily the best strategy for everyone else.
[...] Bain & Company management consultant Jue Wang said many of the big businesses her firm advises have been taking a closer look at returns on their AI investments. "The token cost for them has been doubling, almost every other month," she said. "Let's say $200 per developer per month. Multiply that by 20,000 developers, which is often what we're dealing with at these companies, and that quickly gets you to a number that is not a line item that any general manager has planned for." Sometimes that just means not using the AI equivalent of a sledgehammer to crack a nut. "Not everything needs a Claude Opus 4.6," she said of one of Anthropic's more capable models suited to software engineering or deep research. "And yet you see so many companies, so many users, default to using Opus for everything, including generating emails." That's led to a search for tools that do AI "model routing" -- in which easier queries get automatically sent to cheaper and more efficient AI systems and more complex tasks go to more powerful models.
[...] At the same time, those who favor racking up as many tokens as possible are having a field day with new open-source models from Chinese startups like Moonshot's Kimi or Zhipu's GLM, which nearly match the capabilities of top U.S. models at a fraction of the price. "There is some validity to the theory that this could push tokenmaxxing a little bit further," said Raffi Krikorian, the chief technology officer at Mozilla. "But if we look at the industry overall, I think it's realizing that tokenmaxxing is a dumb thing." It's similar, Krikorian said, to how software companies once considered how many lines of code a programmer wrote to be a good metric of productivity. That later fell out of favor. "I think tokenmaxxing is moving through the exact same pattern," he said. "I think this is going to be an interesting blip that we're all going to look back to laugh at in a year."