While the PCI-SIG has announced that the release of the PCI Express® 6.0 (PCIe 6.0) specification should arrive in 2022, Rambus is already addressing the needs of early adopters looking for the most advanced PCIe 6.0 IP solutions for their SoC and ASIC designs.
You can find all about the new generation specification in the article below.
What is PCIe 6.0?
Since PCIe 3, each new generation of the standard has seen a doubling in the data rate. PCIe 6.0 will boost the data rate to 64 gigatransfers per second (GT/s), twice that of PCIe 5.0. For a x16 link, which is typical of graphics and network cards, the bandwidth of the link reaches 128 gigabytes per second (GB/s). As in previous generations, the PCIe 6.0 link is full duplex, so it can deliver that 128 GB/s bandwidth in both directions simultaneously.
PCIe is proliferated widely beyond servers and PCs, with its economies of scale making it attractive for data-centric applications in IoT, automotive, medical and elsewhere. That being said, the initial deployments of PCIe 6.0 will target applications requiring the highest bandwidth possible and those can be found in the heart of the data center: AI/ML, HPC, networking and cloud graphics.
The following chart shows the evolution of the PCIe specification over time:
What’s new with PCIe 6.0?
To achieve the 64 GT/s, PCIe 6.0 introduces new features and innovations:
1. PAM4 Signaling:
On the electrical layer, PCIe 6.0 uses PAM4 signaling (“Pulse Amplitude Modulation with four levels”) that combines 2 bits per clock cycle for 4 amplitude levels (00, 01, 10, 11) vs. PCIe 5.0, and earlier generations, which used NRZ modulation with 1 bit per clock cycle and two amplitude levels (0, 1).
- Comparison of NRZ modulation and PAM4 modulation
2. Forward Error Correction (FEC)
There are always tradeoffs, and the transition to PAM4 signal encoding introduces a significantly higher Bit Error Rate (BER) vs. NRZ. This prompted the adoption of a Forward Error Correction (FEC) mechanism to mitigate the higher error rate. Fortunately, the PCIe 6.0 FEC is sufficiently lightweight to have minimal impact on latency. It works in conjunction with strong CRC (Cyclic Redundancy Check) to keep Link Retry probability under 5×10-6. This new FEC feature targets an added latency under 2ns.
While PAM4 signaling is more susceptible to errors, channel loss is not affected compared to PCIe 5.0 due to the nature of the modulation technique, so the reach of PCIe 6.0 signals on a PCB will be the same as that of a PCIe 5.0.
3. FLIT Mode:
PCIe 6.0 introduces FLIT mode, where packets are organized in Flow Control Units of fixed sizes, as opposed to variable sizes in past PCIe generations. The initial reason for introducing FLIT mode was that error correction requires working with fixed size packets; however, FLIT mode also simplifies data management at the controller level and results in higher bandwidth efficiency, lower latency, and smaller controller footprint. Let’s address bandwidth efficiency for a minute: with fixed-size packets, the framing of packets at the Physical Layer is no longer needed, that’s a 4-byte savings for every packet. FLIT encoding also does away with 128B/130B encoding and DLLP (Data Link Layer Packets) overhead from previous PCIe specifications, resulting in a significantly higher TLP (Transaction Layer Packet) efficiency, especially for smaller packets
4. Other changes in PCIe 6:
- L0p mode – enabling traffic to run on a reduced number of lanes to save power
- A new PIPE specification – for the PHY to Controller interface
- Many new optional features we will explore in future articles
PCIe 6.0 Fun Fact: the x32 and x12 interface widths from earlier generations are dropped. While in PCIe 5.0 and earlier specifications, these widths were never implemented in the market.
Why PCIe 6.0 now?
Before 2015, the PCIe was well ahead of the market in terms of available bandwidth required for use cases. There wasn’t a market driver to run faster, thus the slower evolution of the specification. After 2015, global data traffic has exploded. Data centers transitioned to 100G Ethernet (and up) pushing the bottleneck to the PCIe interconnects in servers and network devices.
The PCIe 6.0 specification fully supports the transition to 800G Ethernet in data centers: 800 gigabit per second (Gb/s) requires 100 GB/s of bandwidth which falls within the 128 GB/s envelope of a x16 PCIe 6.0 link; 800G Ethernet, like PCIe, is full duplex. Further, data center general compute and networking are not the sole driving forces behind PCIe 6.0. AI/ML accelerators have an insatiable need for more bandwidth. Processing AI/ML training models is all about speed, and the faster accelerators can move data in and out, the more efficient and cost effective the training can be executed.
PCIe is everywhere in modern computing architectures, and we expect PCIe 6.0 will gain quick adoption in performance-critical applications in AI/ML, HPC, cloud computing and networking.
For designs for the upcoming generation of PCIe, Rambus offers the PCIe 6.0 Controller which is backwards compatible to PCIe 5.0, 4.0 and 3.1/3.0. And for PCIe 5.0 and previous generations, Rambus offers completed interface subsystems (PHY and Controller) for easy integration into SoCs and ASICs.
Explore more primers:
– PCI Express 5 vs. 4: What’s New?
– DDR5 vs DDR4 – All the Design Challenges & Advantages
– Hardware root of trust: All you need to know
– Side-channel attacks: explained
– Compute express link: All you need to know
– MACsec Explained: From A to Z
– The Ultimate Guide to HBM2E Implementation & Selection