IBM and Together AI Invest $240 Million in Open-Source AI Inference Cluster

IBM and Together AI have entered into a multi-year, $240 million agreement to build a large-scale AI inference cluster on IBM Cloud. The initiative is designed to provide enterprises with a flexible, cost-effective alternative to closed proprietary AI systems by hosting open-source models.

To power the cluster, the two companies are investing in Nvidia HGX B300 systems. The infrastructure is based on Nvidia's Blackwell architecture and incorporates Spectrum-X Ethernet networking, which has been specifically tuned to maximize inference efficiency.

The Shift Toward Inference Economics

This partnership arrives as the AI industry undergoes a fundamental shift in priority. While early competition focused on the prestige of training the largest possible models, the market is now pivoting toward the economics of running them—a process known as inference.

Many enterprises are increasingly drifting toward open-source models to reduce operational costs and enhance security. By utilizing open-source frameworks, companies can inspect the models they use and host them on infrastructure they control, effectively avoiding the vendor lock-in associated with closed systems like those provided by OpenAI or Anthropic.

Strategic Implications for IBM

For IBM, the move is a strategic attempt to compete with hyperscalers such as Amazon, Microsoft, and Google. Rather than attempting to match the sheer cloud scale of these giants, IBM is focusing on economic efficiency and the flexibility of open-source ecosystems.

By positioning IBM Cloud as a hub for affordable, sovereign AI infrastructure, the company is targeting high-stakes sectors such as healthcare and banking. These industries typically require strict data control and residency, making the ability to host models on controlled infrastructure a critical requirement.

Future Outlook

As the cluster comes online, the industry will be watching to see if this specialized focus on inference efficiency can successfully lure enterprises away from the dominant proprietary platforms. The success of the venture depends on whether the cost savings and security benefits of open-source hosting outweigh the convenience of integrated, closed-loop AI services.

Ultimately, the $240 million investment signals IBM's conviction that the future of enterprise AI lies in transparency and infrastructure sovereignty. By prioritizing the inference layer, IBM aims to capture the growing segment of the market that values control over convenience.