IBM has signed a $240 million deal with Together AI to build a large-scale inference cluster, a move that puts the tech giant deeper into the business of running AI models rather than just training them. The agreement, announced this week, is aimed at improving performance and cutting costs for enterprises that rely on AI in the cloud.
What the deal covers
The deal centers on constructing an inference cluster — the infrastructure used to run trained AI models in production. Together AI, a company that provides cloud services for AI workloads, will work with IBM to deploy this system. The $240 million price tag makes it one of the larger infrastructure commitments IBM has made in the AI space recently.
Neither company disclosed a timeline for when the cluster will go live, but the scale suggests it's meant for heavy, continuous use. Inference is the stage where AI models actually answer queries, generate text, or process images, and it's often more demanding on hardware than training because it happens in real time.
Why inference is the new battleground
For years, the AI industry focused on training ever-larger models. But as those models move into production, the cost of running them becomes the bottleneck. Inference clusters are designed to handle that load efficiently, and IBM's bet is that enterprises will pay for speed and reliability.
The company has been pushing its watsonx platform and cloud services, but this deal signals a more direct play in the infrastructure layer. By partnering with Together AI, IBM gets access to specialized expertise in optimizing inference workloads, something that's become a differentiator in the crowded cloud market.
Cost and performance at the center
The stated goal is to enhance performance and cost efficiency. That's a practical pitch for businesses that have watched their AI bills climb as they scale usage. An inference cluster that can process requests faster and use less energy per query could make a real difference in operating expenses.
IBM isn't the only one chasing this. Cloud providers like Amazon, Microsoft, and Google have all built their own inference infrastructure, but IBM's approach with a dedicated partner like Together AI is a bit different. It's a bet that specialization beats a one-size-fits-all cloud offering.
The deal is signed, but the work is just beginning. Building a large-scale inference cluster takes time, and the companies will need to integrate their systems and test the setup before customers can use it. IBM hasn't said which specific models or workloads the cluster will support first, but the expectation is that it'll serve enterprise clients across industries.
For Together AI, the deal is a major validation of its technology. For IBM, it's a chance to prove that its cloud can handle the most demanding AI tasks without breaking the bank. The real test will come when the cluster goes live and customers start measuring the results.
