Rambus announced this week that it demonstrated for the first time a PCI Express 5.0 Controller IP (PCIe 5 Controller) operating at 32 GT/s on a leading FPGA platform.
“We’ve achieved a new industry benchmark with the demonstration of our PCIe 5.0 Controller operating at 32 GT/s on popular FPGA platforms,” said Scott Houghton, general manager of Interface IP at Rambus. “With the importance of FPGAs in markets from defense to the data center, this solution developed by the newly-acquired PLDA team expands the Rambus portfolio and offers the next level of performance for mission-critical applications.”
Beyond the technical feat, you might wonder: what are the benefits for Rambus customers?
The deployment of PCIe 5.0
We are seeing a notable ramp up in the number of ASIC design starts supporting the PCIe 5.0 interface, with mainstream commercial server platforms supporting the standard on the near horizon.
With the adoption of 5G, 400G Ethernet, and data-intensive AI/ML applications in cloud environments, PCIe 5.0 is becoming the de-facto CPU interface for high-end SoCs. Even for ecosystems which have yet to scale to 32 GT/s, SoC vendors are futureproofing their designs by implementing the PCIe 5.0 interface standard, knowing that backwards compatibility ensures their chips will work with older previous generation components.
FPGAs supporting PCIe 5.0 natively as a hardened controller block (aka “hard” controller IP) are slowly appearing, however the availability of PCIe 5.0 controllers as synthesizable “soft” IP on these FPGAs fill several gaps and enable extended use models, as discussed below.
What are the applications for soft PCIe 5.0 Controller IP on FPGAs?
In general, soft PCIe Controller IP is needed when one of the following conditions are met:
- The target FPGA can support the PCIe protocol stack (i.e. has PCIe-capable PHY), but has no built-in PCIe controller
- The target FPGA does not have enough built-in PCIe controller blocks to enable the customer’s application
- The FPGA’s built-in PCIe controller does not have the capabilities or does not expose the features required for the customer’s application
We are seeing use case (#1) at customers who use FPGAs in such volumes that they aim for the lowest cost parts that can support their performance requirements. However, these devices may not have any built-in PCIe controllers. An example of this are customers in the networking space developing smart NICs (100G, 400G) based on FPGAs and requiring PCIe 5.0 to support network bandwidths. The end product volumes justify a medium range, lower cost FPGA solution, which have PCIe 5.0 capable PHYs but no built-in PCIe controllers. In this case, the Rambus Controller IP for PCIe 5.0 fills the gap with the cost of the IP license amortized across the lifetime production.
Rambus is also seeing use case (#2) with customers at the very high end of the spectrum, utilizing the ‘latest and greatest’ FPGA but lacking sufficient PCIe 5.0 controller blocks to build their applications. A typical example would be in the test & measurement sector, where system designers are developing NVMe SSD testers based on FPGAs. These systems have requirements for multiple downstream PCIe 5.0 ports, in order to connect multiple SSDs for simultaneous testing. In that case, the Rambus PCIe 5.0 Controller IP supplements the FPGA’s available built-in PCIe 5.0.
Use case (#3) are also very common, as the FPGA built-in PCIe 5.0 controller may not provide the right set of features for the application. There are several applications in test & measurement, HPC, storage & networking and other high-performance applications that require bridging between PCIe and other protocols (standards based or proprietary) at high speed. In these cases, the Rambus PCIe 5.0 controller IP will usually provide more flexibility than hard controller IP blocks. This includes offering a variety of dedicated hooks and sideband interfaces. Not to mention, the Rambus controller IP can be customized on- demand to fit specific needs, something that hard IP blocks do not allow.
Why was demonstrating soft PCIe 5.0 Controller IP on FPGAs a challenge?
Each new version of the PCIe specification pushes protocol to higher data rates. If moving up to PCIe 5.0 speeds is complex for ASICs, it is even harder for FPGAs. The challenge is to manage protocol layers with IP blocks running at up to 500 MHz while reaching 32 GT/s signaling at the PHY level.
This is accomplished using the FPGA SerDes block connected to a custom-designed PCS that manages PCIe 5.0 speed and link width. The PCS provides a standard PIPE interface that matches the PCIe controller supported configurations. Beyond the functional aspects, the trickiest task is to consistently produce a binary that meets timings and to find the correct analog settings for the SerDes.
What’s next with the PCIe 5.0 Controller IP for FPGAs?
In addition to enabling the various use models described earlier, the validation of a PCIe 5.0 Controller IP for FPGAs opens the door for Rambus to develop a variety of prototyping and bring- up solutions to accelerate PCIe 5.0 based designs. This includes:
- The launch of a PCIe 3.0 x16 to PCIe 5.0 x4 interposer (Gen5HOST) that enables testing PCIe 5.0 DUTs at 32 GT/s per lane using a standard PCIe 3.0 motherboard.
- The launch of Inspector for PCIe 5.0 that enables designers to diagnose and debug their PCIe 5.0 DUT (speed, width, detected and active lanes, equalization status, LTSSM state history, etc.)
These solutions had already been developed and released for PCIe 4.0 by the former PLDA team and were highly successful: