# How AWS Graviton5 Delivers 25% Better Performance by Redesigning the Chip Itself
*AWS rebuilt Graviton5 around four chiplets, 192 cores, a 192 MB cache, and faster memory, improving real workloads by up to 35%*
By [Rohit Lakhotia](https://scaleengineer.com/authors/rohit-lakhotia)
Published: 2026-11-09
Canonical: https://scaleengineer.com/blog/how-aws-graviton5-delivers-25-better-performance-by-redesigning-the-chip-itself
---
Every time you open an app, run a database query, or call an AI agent hosted on AWS, a CPU somewhere in a data center does the work\. For years, the simplest way to make that CPU faster was to shrink its transistors and fit more of them on a chip\. That's the promise of Moore's law\.

But shrinking transistors alone doesn't give the gains it used to\. A CPU can have more cores than ever and still spend much of its time doing nothing, because it's waiting for data from memory, guessing wrong about what code runs next, or sending messages across a crowded chip\.

AWS has been building its own Arm\-based processors, called [Graviton](https://en.wikipedia.org/wiki/AWS_Graviton), for eight years\. Graviton now powers over 350 EC2 instance types, running everything from web applications and microservices to databases, ML inference, video encoding, and agentic AI\.

With Graviton5, the fifth generation, AWS didn't just add more cores\. It changed how the chip is put together\. The results:

- 192 cores, double Graviton4's 96
- 25% better performance per core
- Up to 35% faster web applications and ML inference
- Up to 30% faster databases

Graviton5 is now generally available in the new M9g and M9gd EC2 instances\. Let's understand how AWS achieved this in this blog today\.

## Why More Cores Isn't Enough

Imagine a kitchen with 192 chefs\. Hiring more chefs only helps if each one can get ingredients quickly, knows what dish comes next, and doesn't bump into the others\.

CPUs work the same way\. Raw core count matters, but real performance depends on three things:

1. How quickly cores get data, through memory and caches\.
2. How well each core predicts what code runs next, through branch prediction\.
3. How efficiently cores talk to each other and to memory, through the chip's layout\.

AWS points out an important detail here\. Micro benchmarks, which are small loops that test one thing in isolation, behave very differently from real workloads\. A real database has huge amounts of code, unpredictable control flow, and constant memory access\. Graviton5 was designed for the database, not the benchmark loop\.

Let's go through each improvement\.

## Better Branch Prediction: Guessing the Right Path

Programs are full of decisions: `if` statements, loops, and function calls\. Each one is a branch, a point where the code can take more than one path\.

Modern CPUs don't wait to find out which path is correct\. They predict the likely path and start running it ahead of time\. This is called speculative execution\. A correct guess saves time\. A wrong guess means the CPU throws away that work and starts again\.

Graviton5 uses the Neoverse V3 core, designed jointly by Arm and Amazon's Annapurna Labs\. A big improvement in this core is much better branch prediction\. Real applications like databases have complex, branch\-heavy code, so they benefit the most\. Better prediction alone helps databases run up to 30% better\.

## The Memory Problem: 330 Cycles of Waiting

Here's a number that explains a lot about CPU design\.

DRAM, the main memory, can be about 100 nanoseconds away from the CPU\. That sounds tiny\. But a CPU running at 3\.3 GHz completes 3\.3 cycles every nanosecond, so one trip to memory costs about 330 cycles\. For all that time, the core could be stuck waiting\.

CPUs solve this with caches, small and fast memory layers that keep frequently used data close to the cores\. If the data is already in a cache, the CPU skips the long trip to DRAM\.

Graviton5 has three cache levels:

- L1 cache: 64 KB
- L2 cache: 2 MB
- L3 cache: 192 MB, more than five times as much as the previous generation

That much larger L3 cache means far more requests can be served without going to main memory\. When the CPU does need DRAM, Graviton5 supports DDR5\-8800 memory\. AWS worked closely with DRAM manufacturers to reach this speed, and says Graviton instances now have the fastest memory of any processor instances in the cloud\.

## What is a Chiplet?

Traditionally, a processor was built as one large piece of silicon, called a monolithic die\. As chips grow, that gets harder and more expensive to make\.

A chiplet design splits the processor into several smaller dies that work together as one chip\. Each chiplet can hold its own cores and controllers, and they're connected inside the same package\.

Graviton isn't new to this\. Graviton3 was the first Graviton CPU to use chiplets, with seven dies spread across cores, DRAM controllers, and PCIe controllers\. Graviton4 kept the same architecture with a few refinements\.

Graviton5 changes it substantially\.

## The Redesign: Four Self\-Contained Chiplets

In Graviton5, the 192 cores are split across four chiplets\. Each chiplet contains:

- 48 cores
- Its own DRAM controllers
- Its own PCIe controllers

The key difference is that there is no separate I/O die and no separate DRAM controller die\. Each chiplet now holds the cores together with the controllers they need to reach memory and devices\. The chiplets are linked by custom die\-to\-die connections with up to 420 GB/s of bandwidth between them\. This keeps latency between cores in the mesh low, even when they sit on different chiplets\.

## Why This Layout Matters: NUMA and Cache Partitioning

This redesign enables a smarter way to run virtual machines, and that's where NUMA comes in\. NUMA stands for Non\-Uniform Memory Access\. In a NUMA system, a core reaches memory attached to its own region faster than memory attached to another region\. Instead of treating the whole chip as one equal pool, the chip is divided into regions with their own nearby memory\.

Because each Graviton5 chiplet has its own memory controllers, AWS can:

- Configure two or four NUMA regions per chip\.
- Partition the L3 cache to match the size of the VMs running on the CPU\.
- Reduce memory latency for VMs with 48 cores or fewer\.

A VM with 48 cores or fewer can fit inside a single chiplet\. Its cores, cache, and memory controllers are all close together, so it rarely needs to reach across the chip for data\.

## A Smaller Process: 3 Nanometers

Graviton5 also moves to a 3\-nanometer manufacturing process\. Smaller transistors mean more circuits fit in the same space, and signals travel shorter distances across the chip\. That's how Graviton5 doubles the core count while also making each core 25% faster\.

## Raising the Security Bar: The Nitro Isolation Engine

Performance isn't the only upgrade\. The M9g and M9gd instances also introduce the Nitro Isolation Engine\. In the cloud, many customers' virtual machines run on the same physical hardware\. The most important guarantee is isolation: one VM must never be able to access another VM's memory or state\.

The Nitro Isolation Engine is a purpose\-built component of the AWS Nitro System that enforces this isolation\. It mediates all access to:

- VM memory
- CPU register state
- I/O devices

It does this through a minimal set of APIs, which keeps the part that must be trusted small\.

### What is Formal Verification?

Normally, software is checked by testing: run it against many scenarios and confirm it behaves correctly\. But tests only cover the cases someone thought to write\.

Formal verification goes further\. It mathematically proves that hardware or software behaves as intended in all cases, not just the tested ones\.

AWS used formal verification on the Nitro Isolation Engine, making Nitro the first formally verified cloud hypervisor\. Isolation between VMs is now backed by mathematical proof rather than test coverage alone\.

## The Results

Putting it all together, Graviton5 offers up to 25% better computational performance than Graviton4\-based instances\.

For real workloads, the gains are larger:

- Web applications: up to 35% faster
- Machine learning inference: up to 35% faster
- Databases: up to 30% faster

None of these gains come from a single change\. They come from better branch prediction, a much larger cache, faster memory, a smaller process, and a chiplet layout that keeps data close to the cores that need it\.

## The Bigger Picture

Graviton5 shows a clear shift in how processors get faster\. Shrinking transistors still helps, but it's no longer enough on its own\.

The real gains come from architecture: how the chip is split, where memory controllers sit, how much data stays close to the cores, and how well the CPU predicts what comes next\. AWS designed for its customers' real workloads, so the improvements show up where it matters, in databases, web services, and AI inference, not just in benchmark charts\.

And with the Nitro Isolation Engine, AWS is showing that faster cloud hardware can come with stronger, mathematically proven security\.

## Key Takeaways

- Graviton5 doubles the core count from 96 to 192, with each core about 25% faster\.
- The chip uses four chiplets, each with 48 cores and its own DRAM and PCIe controllers, and no separate I/O die\.
- Custom die\-to\-die links provide up to 420 GB/s of bandwidth between chiplets\.
- The L3 cache grows to 192 MB, more than five times the previous generation, which cuts slow trips to DRAM\.
- DDR5\-8800 memory and PCIe gen6 support make it the fastest memory of any cloud processor instance\.
- The Neoverse V3 core improves branch prediction, helping databases run up to 30% better\.
- Two or four NUMA regions and L3 cache partitioning reduce latency for VMs of 48 cores or fewer\.
- The move to a 3 nm process enables higher circuit density and faster on\-chip communication\.
- The Nitro Isolation Engine makes Nitro the first formally verified cloud hypervisor\.

Official blog from Amazon: **[Graviton5’s improved design increases speed and energy efficiency — beyond Moore’s law](https://www.amazon.science/blog/graviton5s-improved-design-increases-speed-and-energy-efficiency-beyond-moores-law)**

By now, you must have had a clear idea of,** How AWS Graviton5 Delivers 25% Better Performance by Redesigning the Chip Itself? **In a nutshell, AWS rebuilt Graviton5 around four self\-contained chiplets with 192 cores, a much larger cache, faster memory, and better branch prediction, giving up to 35% gains on real workloads\.

**Congratulations\! You've just advanced another step in your tech journey\. Keep progressing\!**