‘Gaudi 3 can run AI models at much better value than NVIDIA’s H100’: Intel’s Anil Nanduri

Intel is pitching Gaudi 3 as a more affordable alternative to NVIDIA H100.

Anil Nanduri, VP and GM, DCAI Category, Head of Intel Al Acceleration Office (Image credit: Vivek Umashankar/The Indian Express)

Gaudi 3 is Intel’s third-generation AI accelerator, the successor to Gaudi 2, and the company is pitching it as a direct competition to NVIDIA H100, currently one of the most popular AI accelerators used by all the major AI startups across the world.

“We believe Gaudi 3 can run anything that a big NVIDIA GPU can run in terms of model AI sizes, and at a much better value,” said Anil Nanduri, VP and GM, DCAI Category, Head of Intel Al Acceleration Office.

Nanduri, in an interaction with indianexpress.com at the recently-held Intel Tech Tour in Taipei, Taiwan, also said, “We are not looking at it from a head-to-head performance, we are looking at it from a performance-per-dollar angle.” According to Intel, Gaudi 3, when compared to H100, will offer 1.9 times more performance per dollar while training and 2.3 times more performance while inferencing.

Story continues below this ad

Also read | Computex 2024: Intel unveils Lunar Lake processor for AI PCs, Xeon 6 server processor, and Gaudi 3 AI accelerator

The list price of the Gaudi 3 AI accelerator kit––which includes eight Gaudi 3 accelerators––is set at $1,25,000. NVIDIA H100 AI accelerator kit with eight accelerators, on the other hand, is reported to be over $4,30,000.

Intel CEO showcasing Gaudi 3 silicon (Image credit: Vivek Umashankar/The Indian Express)

Unlike most AI accelerators, instead of a proprietary network, Gaudi 3 uses (RoCE) RDMA over Converged Ethernet, and makes use of open frameworks like PyTorch. Hence, Nanduri calls Gaudi 3 an open platform. With Gaudi 3, Intel also wants to promote more open source models.

Nanduri also confirmed that “Indian AI startup Ola Krutrim,” which recently set up its own AI servers, is “based on Gaudi 2,” and is planning to upgrade to Gaudi 3, considering the higher value offered on the Gaudi 3. “Gaudi 3 is going into production soon. It has already been sampled to select customers, and will be available widely from Q3 2024,” Nanduri said.

When asked about his views on a generalised AI accelerator like a Gaudi 3 and purpose-built AI accelerators from brands like Google and Microsoft––which will be used for specific model training and inferencing––Nanduri said, “We (the industry) is still quite far away from understanding what is the matured hardware and software layer for AI accelerators. There will always be a trade-off between generality and specificity, and a right balance between the two, and these two will probably co-exist.”

Story continues below this ad

The writer is in Taiwan, Taipei, on the Invite of Intel India.