OpenAI is building a software layer designed to run its artificial intelligence workloads across chips from Nvidia, AMD, Broadcom, and other suppliers. The effort, still in development, aims to reduce the company's near-total reliance on Nvidia's hardware and open the door for more chipmakers to compete in the lucrative AI market.
Why diversify chip sources
Nvidia dominates the market for AI training and inference chips. Its GPUs power most large language models, including the ones behind OpenAI's own GPT series. But that single-supplier dependency carries risks. Supply shortages, price hikes, or technical bottlenecks could slow development. By building a hardware-agnostic software layer, OpenAI can tap into a broader range of processors — potentially lowering costs and speeding up model training.
AMD and Broadcom have been pushing their own AI accelerator designs. AMD's Instinct line and Broadcom's custom ASICs for cloud customers are already in use at some data centers. OpenAI's software layer would allow its models to run efficiently on those chips without requiring separate code rewrites or specialized hardware support for each one.
How the layer would work
Details are sparse, but the idea is similar to what Google did with its Tensor Processing Units (TPUs). Instead of building custom chips from scratch, OpenAI is writing a middleman layer — essentially a compiler or runtime — that translates AI instructions into low-level commands that any compatible chip can understand. That would let the company treat different silicon as interchangeable parts, at least for inference.
Training is tougher. Nvidia's CUDA platform and its tightly integrated software stack make switching hard. OpenAI would need to ensure its layer performs comparably on AMD's ROCm software or Broadcom's proprietary toolkits. Early reports suggest the company has been testing its models on AMD hardware internally, but no timeline for a public rollout has been announced.
Potential market impact
If the software layer works well, it could shake up the AI chip market. Nvidia currently commands an estimated 80-90% of the data center AI chip business. A portable software layer would make it easier for competitors to win business from large AI labs. Cloud providers like Amazon Web Services, Google Cloud, and Microsoft Azure all offer AMD- or Broadcom-based instances; OpenAI's layer could give customers more choices, potentially lowering cloud costs.
But the flip side is execution risk. Building a universal abstraction layer for such complex, high-performance workloads is hard. GPUs from different vendors have different memory architectures, threading models, and optimization quirks. One wrong abstraction could kill performance, negating the whole point. OpenAI's team is reportedly still wrestling with those issues.
What comes next
OpenAI has not set a formal launch date for the software layer. The company is still finalizing its design and testing it across multiple chip types. Whether it can deliver a production-ready version that matches Nvidia's raw speed — and at what development cost — remains the open question.
