The battle for supremacy in AI computing is heating up as AMD and Intel set their sights on the fast-expanding AI inferencing market, positioning themselves to challenge Nvidia’s long-standing dominance. AI inferencing, the process of using trained models to make real-time decisions or predictions, is emerging as a pivotal segment with vast revenue potential.

The Shift Towards AI Inferencing

AI inferencing is increasingly viewed as the stage where businesses reap tangible benefits from their AI investments. Unlike AI training, which relies heavily on Nvidia’s high-end GPUs, inferencing demands cost-effective and energy-efficient solutions for scaling applications. AMD and Intel are capitalizing on this shift, offering alternatives that are significantly cheaper and optimized for real-time tasks.

Sunil Gupta, CEO of Yotta, highlights this trend:

“AI inferencing will become a larger market than training over time, and both AMD and Intel are positioning their GPUs and CPUs to capitalize on this transition.”

AMD’s Strategy: MI300 and Beyond

AMD is leveraging its MI300 GPU series, designed for high memory capacity and bandwidth efficiency, to gain traction in inferencing. AMD CEO Lisa T. Su emphasized the company’s growing focus on this segment, projecting $5 billion in data center GPU revenue for 2024, driven largely by early success in inferencing.

AMD’s competitive pricing—offering alternatives at a fraction of Nvidia’s costs—has caught the attention of enterprises aiming to scale AI applications without breaking their budgets.

Intel’s Unique Advantage: CPU Expertise

Intel is tapping into its expertise in CPUs to create niche solutions for AI inferencing. With its focus on power-efficient hardware and integration capabilities, Intel is carving out a position in edge computing applications such as autonomous vehicles, IoT systems, and real-time analytics.

Jeongku Choi, an analyst at Counterpoint Research, notes:

“Intel and AMD can leverage their power-efficient, cost-effective hardware to challenge Nvidia in this space.”

Inferencing: The Future of AI Workloads

Inferencing is expected to surpass training in workload demands as AI applications become more pervasive. Gartner projects that by 2028, over 80% of workload accelerators in data centers will focus on AI inferencing, compared to 40% in 2023.

Applications such as real-time traffic analysis in autonomous vehicles, personalized recommendations on mobile devices, and IoT-driven analytics are driving the demand for inferencing hardware closer to end users.

Arun Moral, Managing Director at Primus Partners, explains:

“Once we cross 5 trillion data points, the scope for training diminishes. This makes inferencing the future growth driver for AI hardware.”

Nvidia’s Response

Nvidia is not sitting idle. The company has expanded its portfolio with ARM-based CPUs and optimized GPU platforms tailored for inferencing. Jensen Huang, Nvidia’s CEO, underscores the complexity of this market:

“Inferencing requires high accuracy, low latency, and high throughput simultaneously, making it very challenging to build efficient systems. But we are seeing significant growth in this area.”

Conclusion

The AI inferencing market is shaping up to be the next frontier in the race for AI dominance. While Nvidia maintains a stronghold with its data center GPUs, AMD and Intel are poised to disrupt the status quo with cost-effective and power-efficient alternatives.

As AI applications continue to proliferate across industries, the competitive dynamics in inferencing hardware will play a crucial role in determining the next leader in AI computing.