AT&T adopts 'tokenomics' to curb surging AI operational costs
The telecom giant is shifting workloads to open-source models and proprietary routing to manage a fivefold increase in token volume.
AT&T has implemented a comprehensive "tokenomics" strategy to manage the escalating costs of generative AI as its operational scale explodes. The move comes as the company's daily AI token volume surged from 8 billion to more than 45 billion, necessitating a shift away from expensive proprietary models toward a more sustainable orchestration layer.
To stabilize spending, AT&T is aggressively migrating its AI workloads. The company now aims to serve between 65% and 70% of its total AI tokens through open-source models. Central to this effort is the development of OTel 1.0 and 2.0, which are specialized iterations of Google's Gemma 4 model fine-tuned specifically for the telecommunications industry. To further optimize efficiency, AT&T is identifying generative AI tasks—specifically simple classification—and converting them back to traditional machine learning to avoid burning tokens entirely.
The Logic of Migration
This strategic pivot is driven by the observation that the gap between "frontier" proprietary models and open-weight alternatives narrows over time. AT&T is specifically targeting use cases that are more than a year old, operating on the premise that open models have typically caught up in capability for those specific functions. By migrating these legacy tasks, the company can maintain performance while significantly lowering the cost per request.
Intelligent Orchestration
Beyond model selection, AT&T has built a technical layer to prevent waste. The company utilizes a proprietary "smart router" integrated within a LiteLLM gateway. This router is "cache-aware," a critical feature designed to maintain context across sessions. According to Mark Austin, VP in AT&T’s Data Office, switching models typically results in the loss of the cache, which is a significant financial blow because cached data costs only one-tenth as much as fresh tokens. "We’re trying to keep our overall cost in control," Austin stated.
Industry Implications
AT&T's approach provides a blueprint for other enterprises facing the "scaling wall" of generative AI. It demonstrates that sustainable AI deployment is not about relying on a single, massive model, but about building an intelligent routing layer that matches the task to the most cost-effective tool. By combining fine-tuned open models with cache-aware routing, the company is proving that performance does not have to be sacrificed for fiscal discipline.
What's Next
As AT&T continues to refine its OTel models, the industry will be watching to see if this hybrid approach—mixing open-source, proprietary, and traditional ML—becomes the standard for Fortune 500 AI deployments. While the company has seen success in targeted applications, the long-term impact on overall operational expenditure remains the primary metric for the success of this tokenomics experiment.