AMD introduces Epyc 6 processors for AI agent workloads

Advanced Micro Devices Inc. today debuted a new line of central processing units for servers.

The Epyc 6 series, which is also known as Venice, comprises 10 CPUs that are organized into two groups. The first set includes chips with a large number of relatively slow cores. The remaining CPUs feature fewer, higher-frequency cores. Both chip collections feature optimizations designed to speed up artificial intelligence agents.

AI agents run their neural networks on graphics cards and use CPUs to power their tools, the applications with which they carry out work. CPUs also perform certain other tasks in machine learning clusters. According to AMD, Epyc 6 chips can support up to 3.3 more agents per watt than Nvidia Corp.’s Vera CPU. Additionally, the company says that its processor series significantly outperforms comparable Intel Corp. silicon.

Deploying an AI agent’s tools on CPUs with a relatively small core count can reduce software expenses in some cases. Certain enterprise applications, notably databases, are billed according to the number of cores on which they run. As a result, deploying a database on one high-frequency core is cheaper than using two slower ones that together provide the same amount of processing power.

The two highest frequency Epyc 6 chips have a base speed of 4.05 GHz. They can boost their clock to 5.3 GHz when running demanding workloads. Both chips have 96 cores and a 384-megabyte cache.

The two densest Epyc 6 chips, meanwhile, include 256 cores with a base speed of 2.55 GHz. Each core can power two threads, or stream of calculations, for a total of 512 threads. That enables the chips to run a large number of AI agents at once. Some large language models spin up hundreds of agents or more when tackling complex tasks.

The Epyc 6 series is based on a new core architecture called Zen 6. Its IPC, a key measure of CPU performance, is 31% higher than AMD’s previous design when running AI workloads. The increase partly stems from the fact that AMD uses Taiwan Semiconductor Manufacturing Co.’s latest two-nanometer node to make Zen 6 cores.

The cores come in two editions. There’s a version called simply Zen 6 that is optimized for clock speed and a scaled-down edition, Zen 6c, that is designed to power CPUs with high core counts. Zen 6c core features an enhanced out-of-order processing mechanism not included in the standard edition. The mechanism speeds up programs by performing some calculations well before the results are needed.

Zen 6 also includes AI-specific optimizations. The architecture is compatible with the FP16 data format, which makes it possible to lower agents’ hardware footprint by compress the information they process. There’s also improved support for VNNI-INT8, a technology that condenses multiple AI calculations into a single operation to save time. 

“AI is significantly expanding the role of the CPU across every layer of the data center,” said Dan McNamara, the senior vice president and general manager of AMD’s Compute and Enterprise AI unit. “With 6th Gen AMD EPYC, customers can standardize on the proven x86 foundation while choosing a workload-optimized solution to run more agents.”

AMD says that a number of major data center suppliers have already started shipping servers equipped with Epyc 6 chips. Additionally, the processors power a pair of Microsoft Azure instances optimized for AI inference workloads and chip design software. 

Image: AMD