Rambus

At Rambus, we create cutting-edge semiconductor and IP products, spanning memory and interfaces to security, smart sensors and lighting.

Home > Blogs

Complete Memory Interface Solution for HBM2E Launched

by Leave a Comment

Rambus has announced a comprehensive interface solution for HBM2E memory consisting of co-verified PHY and memory controller. Operating at a top speed of 3.2 Gbps over a 1024-bit wide interface, the interface can deliver 410 GB/s of bandwidth with a single HBM2E DRAM stack.

HBM2E Memory System with Single DRAM Stack
HBM2E Memory System with Single DRAM Stack

In addition to the speed jump from 2.0 to 3.2 Gbps for HBM2E vs. HBM2, this latest iteration of the HBM standard now supports 12-high DRAM stacks of up to 24 Gb devices, providing an aggregate stack capacity of 36 GB. With 3D stacking of memory, HBM2E provides high bandwidth and high capacity at low power and low latency in a very small footprint.

For AI/ML, HBM2E delivers an outstanding combination of features. AI training capabilities are growing at 10X per year with rapidly expanding training set sizes. With AI/ML training running in data centers that are increasingly space and power constrained, HBM2E’s compact size and low power, in combination with its unmatched bandwidth and capacity, make it an ideal solution.

HBM2E Memory Interface Subsystem Example
HBM2E Memory Interface Subsystem Example

The Rambus co-verified PHY and controller greatly reduce integration complexity for chip designers. Both PHY and controller are fully compliant with the JEDEC JESD235B standard and are backward compatible to HBM2. While together they are an ideal complete solution, both PHY and controller can be paired with JESD235B-compliant 3rd-party solutions if so desired.

This latest portfolio addition highlights Rambus leadership in high-speed memory PHY and controller IP, leveraging the company’s long tradition of signal and power integrity expertise coupled with the design talent and products from recently acquired Northwest Logic. Get all the details on the HBM2E interface solution here.

Download HBM2E and GDDR6: Memory Solutions for AI

Memory Systems for AI: Part 2

by Leave a Comment

In part one of this series, we discussed how the world’s digital data is growing exponentially, doubling approximately every two years. In fact, there’s so much digital data in the world that artificial intelligence (AI) is practically the only way to begin to make sense of it all in a timely fashion. Insights gleaned from digital data are becoming more valuable, and one side effect is the need for greater security to protect the data, the AI models, and the infrastructure as well. Not surprisingly, the increasing value of the data and insights is causing AI developers to want to create more sophisticated algorithms, larger datasets, and new use cases and applications.

The challenge? Everyone wants more performance. However, the semiconductor industry can no longer fully rely on two important tools – Moore’s Law and Dennard (power) scaling – that have powered successive generations of silicon for the past several decades. Moore’s Law is slowing, while Dennard Scaling broke down around 2005. Nevertheless, the explosion of data and the advent of new AI applications are challenging the semiconductor industry to find new ways to provide better performance and better power efficiency.

At the same time, the architecture of the internet is also steadily evolving. We’re all familiar with the cloud-based model where we have endpoints like our phones capturing data. The data is collected and transmitted up into the cloud where servers process and interpret the data, sending some actions back to these devices that can tell us how to navigate and complete other tasks. Looking beyond the cloud, the upcoming rollout of 5G infrastructure will enable processing in a new location: the edge. The edge encompasses a range of locations including base stations and more geographically distributed processing locations that sit between cloud data centers and the endpoint. Edge locations offer the ability to process data closer to where it’s being created and provides promise for applications like autonomous driving that rely on low latency. So, in addition to being able to do some processing in the cloud on data captured at endpoints, we’re going to have another location, the edge, which will change the dynamic.

However, so much data is now being captured that it really can’t all be moved efficiently to the cloud. This is because the growth of the world’s digital data is increasing faster than the speed at which networks are improving. So, the question is: what do you do when you need to process all this data? The ability to process data at the edge provides a convenient way to take the growing amount of digital data, process it in part in locations closer to the endpoints, and then send a smaller amount of higher-value data to cloud data centers, easing the demand on network bandwidth. Processing data in this way offers the potential to improve performance as well as power efficiency.

Of course, the cloud will still be an important element of the data economy even as edge points become more popular with the advent of 5G. We expect the cloud will be used for some of the toughest interpretation and analysis tasks. In some cases, data captured by the endpoints will be sent to the edge where it can be partially processed before being sent to cloud data centers, in other cases the data will be sent directly to the cloud to be aggregated with other data captured across wide geographies.

We expect to see AI deployed across the evolving internet in data centers, endpoints, and at edge locations in-between, although the requirements and implementation will differ in each place. For example, we expect to see the highest performance solutions remaining in the cloud and being wall-plugged. In terms of memory requirements, these deployments will require the highest-performance, lowest-power solutions to support training as well as inferencing. The solutions of choice that we expect to see here are on-chip memory, as well as very high-performance discrete DRAM solutions like HBM and GDDR. Endpoints will still mostly be comprised of battery-powered devices. Much of what we see today – where such devices are typically used for inferencing – is going to be the same in the future. Low-power inferencing can be addressed by on-chip memories as well as low-power mobile memories.

It should also be noted that the definition of the edge has bifurcated and now comprises the near edge and the far edge. The near edge is closer to the cloud, while the far edge is closer to the endpoints. We expect to see a full range of memory solutions spanning the near and far edge. Specifically, at the near edge, closest to the cloud, we expect to see solutions and memory systems that look more like what you see in the cloud, such as on-chip memory, HBM and GDDR. At the far edge, we expect to see memory solutions that are similar to those deployed in endpoint devices, including on-chip memories, LPDDR, and DDR.

Watch GDDR6 and HBM2E: Memory Solutions for AI

Memory Systems for AI: Part 1

by Leave a Comment

Written by Steven Woo for Rambus Press

There has been quite a lot of recent news about domain-specific processors that are being designed for the artificial intelligence (AI) market. Interestingly, many of the techniques used today in modern AI chips and applications have actually been around for several decades. However, neural networks didn’t really take off during the last wave of interest in AI that spanned the 1980s and 1990s. The question is why.

Source: Adapted from Jeff Dean, "Recent Advances in Artificial Intelligence and the Implications for Computer System Design," HotChips 29 Keynote, August 27
Source: Adapted from Jeff Dean, “Recent Advances in Artificial Intelligence and the Implications for Computer System Design,” HotChips 29 Keynote, August 27

The chart above provides some insight as to why AI technology remained relatively static for so many years. Back in the 1980s and 1990s, processors (CPUs) simply weren’t fast enough to adequately handle AI applications. In addition, memory performance wasn’t yet good enough to enable neural networks and modern techniques to displace conventional approaches. Consequently, conventional approaches remained popular in the above-mentioned timeframe.

However, if you fast forward a few decades to today, Moore’s Law has provided us with approximately five orders of magnitude better processing power and about three to four orders of magnitude better memory performance and capacity. These advances have enabled specialized neural network processors to outperform the conventional approaches. As such, neural networks have become the method of choice for a wide range of AI applications.

The timing is indeed fortuitous, as even by the most conservative estimates the world’s digital data is doubling about every two to three years. There’s so much digital data in the world that AI is almost the only way to make sense of it all in a timely fashion. So, there is a dependence of sorts that is forming. The abundance of digital data helps us train our neural networks and make them even better. In turn, AI is rapidly becoming one of the only ways to rapidly process and interpret the data. This trend will continue well into the future, as the current wave of AI activity is just beginning. There is so much more left to do.

For example, many AI developers want to create more sophisticated algorithms, larger datasets and other methods and applications. The challenge? Everyone wants more performance. However, the semiconductor industry can no longer fully rely on two important tools – Moore’s Law and Dennard (power) scaling – as it has for the past several decades. Moore’s Law is slowing, while Dennard Scaling broke down around 2007. Nevertheless, the explosion of data and advent of new AI applications are challenging the semiconductor industry to provide better performance and better power efficiency.

In this “Memory Systems for AI” blog series, we’ll describe how AI is driving a fascinating renaissance in computer architecture by serving as a catalyst for the design of new domain-specific silicon. It’s an extremely exciting time to be in the semiconductor industry because of the resurgence of interest in new AI architectures. And since memory bandwidth is a critical resource for AI applications, memory systems have once again become a hot topic within the semiconductor industry and beyond.

Download HBM2E and GDDR6: Memory Solutions for AI

Cable Haunt vulnerability can give hackers remote access to approximately 200 million cable modems

by Leave a Comment

Danish cybersecurity experts Alexander Krog, Jens Stærmose and Kasper Terndrup of cybersecurity firm Lyrebirds ApS, and independent Danish researcher Simon Sillesen, announced a new vulnerability dubbed Cable Haunt. In the announcement, they disclosed a critical remote code execution vulnerability in hundreds of millions of cable modems.

According to the website the researchers set up, “The vulnerability enables remote attackers to execute arbitrary code on your modem, indirectly through an endpoint on the modem. Your cable modem is in charge of the internet traffic for all devices on the network. Cable Haunt might therefore be exploited to intercept private messages, redirect traffic, or participation in botnets.”

The researchers further state that the affected modems are vulnerable to a DNS rebind attack followed by overflowing the registers and executing malicious functionality. Use of default credentials and a programming error in the spectrum analyzer also contribute to the vulnerability.

While a buffer overflow exploit would normally be written directly to the memory stack, the memory structure of the MIPS assembly language that runs the spectrum analyzer requires the attack code to know the precise memory address of the vulnerable code. To get around this, Cable Haunt uses return-oriented programming to move between pre-existing pieces of code and then create a patchwork of existing code.

This approach indicates that through the use of a hardware root of trust model, this vulnerability could have been eliminated.

Bart Stevens, senior director of product marketing for Rambus elaborates. “The recent Cable Haunt exploit demonstrates the need for a security by design approach, led by a hardware root of trust to silo away secure processes from the main CPU. This siloed approach to security ensures that a potential compromise of the main processor does not expose critical keys and credentials — or impair the execution of security applications that monitor system operation and detect tampering. A hardware root of trust would not necessarily prevent the host CPU from being attacked by a similar attack like Cable Haunt. But at a minimum, a hardware root of trust could prohibit alternate firmware from replacing the original firmware. In addition, many future vulnerabilities could be prevented if sensitive configuration and network parameter handling is moved from the normal host CPU onto a separate and dedicated, secure CPU residing inside the root of trust section.”

It is important to note that this is a proof-of-concept exploit and has not been seen in the wild. The exploit is complicated by the fact that the vulnerable spectrum analyzer component is available on the cable modem’s internal network, and not directly exposed to the internet. While it would require a lot of skill, and maybe a bit of luck, if successful it would give a hacker intimate access to all the data coming in and going out. In a high-value target, the value of that access could inspire some to try.

Additional Resources:
“Hundreds of millions of Broadcom-based cable modems at risk of remote hijacking, eggheads fear,” The Register, 1/10/2020

“Hundreds of millions of cable modems are vulnerable to new Cable Haunt vulnerability,” ZDNet, 1/10/2020

“Cable Haunt Vulnerability Exposes Modems to Remote Attacks,” Tom’s Hardware, 1/13/2020

“Cable Haunt vulnerability affects millions of Broadcom cable modems,” Security Boulevard, 1/13/2020

PCI Express 5 vs. 4: What’s New? [Everything You Need to Know]

by Leave a Comment

Introduction

What’s new about PCI Express 5 (PCIe 5)? The latest PCI Express standard, PCIe 5, represents a doubling of speed over the PCIe 4.0 specifications.

We’re talking about 32 Gigatransfers per second (GT/s) vs. 16GT/s, with an aggregate x16 link bandwidth of almost 128 Gigabytes per second (GBps).

This speed boost is needed to support a new generation of artificial intelligence (AI) and machine learning (ML) applications as well as cloud-based workloads.

Want to know a little more about AI/ML applications and cloud-based workloads?

Both are significantly increasing network traffic. In turn, this is accelerating the implementation of higher speed networking protocols which are seeing a doubling in speed approximately every two years.

You can find much more about PCIe 5 in the article below.

Table of contents

1. PCI Express: Frequently Asked Questions (FAQ)
2. PCIe 5 – A New Era
3. PCIe 5 vs. PCIe 4 (+Comparison table included)
4. PCIe 5: Applications and Market Adoption
5. Complete PCIe 5 Interface Solutions from Rambus
6. Conclusion

 

PCI Express: Frequently Asked Questions (FAQ)

Let’s answer five frequently asked questions about PCI Express and PCIe 5.

a. What is PCI Express 5?

With the preliminary specification announced in 2017, PCIe 5 is a high-speed serial computer expansion bus standard that moves data at high bandwidth between multiple components. The PCIe 5.0 specification was formally released in May of 2019.

You might be wondering why a new PCI Express standard like PCIe 5 is needed. Well, PCIe 5 offers twice the data transfer rate of its PCIe 4 predecessor, delivering 32GT/s vs. 16GT/s. This speed increase is critical to support new AI/ML applications and cloud-centric computing.

b. Why both GT/s and GBps?

GT/s is a measure of raw speed – how many bits can we transfer in a second. The data rate, on the other hand, has to take into consideration the overhead for encoding the signal. Bandwidth is data rate times link width, so encoding overhead’s impact on the data rate translates directly to an impact on bandwidth.

Back in the days of PCIe 2, the encoding scheme was 8b/10b, so there was a hefty overhead penalty for encoding. With such a high overhead, it was particularly useful to have measures of transfer rate (x GT/s) and data rate (y Gbps), where “y” was only 80% of “x.”
With Gen 3 and continuing through to the present Gen 5, the PCI Express standard moved to a very efficient 128b/130b encoding scheme, so the overhead penalty is now less than 2%. As such, the link speed and the data rate are roughly the same.

For a PCI 5 x8 link, 32GT/s raw speed translates to 31.5GBps bandwidth (we chose a x8 link so we could go straight from bits to bytes). And since PCIe is a duplex link, total aggregate bandwidth rounds to 63GBps (32GT/s x 8 lanes / 8 bits-per-byte x 128/130 encoding x 2 for duplex).

c. What is a PCI Express lane?

So what’s a PCI Express lane? Well, a PCIe lane consists of four wires to support two differential signaling pairs. One pair transmits data (from A to B), while the other receives data (from B to A). Want to know the best part? Each PCIe lane is designed to function as a full-duplex transceiver which can simultaneously transfer 128-bit data packets in both directions.

d. What does PCIe x16 mean?

We’ve discussed lanes, but what do they have to do with x16? Well, the term “PCIe x16” is used to refer to a 16-lane link instantiated on a board or a card. Physical PCIe links may include 1, 2, 4, 8, 12, 16 or 32 lanes. The 32-lane link is a pretty rare beast, so in practical terms the x16 represents the top end of the PCI Express link options.

e. What is PCI Express used for?

We’ve talked a lot about PCIe 5, but what is PCI Express actually used for?

You can think of the PCIe interface as the system “backbone” that transfers data at high bandwidth between various compute nodes.What’s the bottom line? Put simply, PCIe 5 rapidly moves data between CPUs, GPUs, FPGAs and ASIC accelerators using links with various lane widths configured to meet the bandwidth requirements for the linked devices.

PCIe 5: A New Era

Now that we’ve explored some PCI Express basics, we can take a closer look at PCIe 5 which was formally released in May of 2019.

PCIe 5 offers long-awaited capabilities for system designers building new cloud-based and AI/ML applications.

You might be wondering how and why.

Well, it starts with PCIe 5 offering twice the bit transfer rate of its PCIe 4 predecessor at 32GT/s vs. 16GT/s. With a x16 link, that scales to ~128GBps aggregate bandwidth.

Here’s another important point: The higher transfer rates of PCIe 5 enables system designers to balance bandwidth requirements with simplicity by working with fewer lanes for certain applications such as 40GigE and SSDs. More demanding, high-bandwidth applications such as AI and ML can take full advantage of a x16 lane.

Want to know what else PCIe 5 technology offers system designers? PCIe 5 is expected to be the technological cornerstone of additional interconnects. For example, Intel recently announced that its upcoming Compute eXpress Link (CXL) cache coherent interconnect will use the PCIe 5 standard’s physical layer (PHY) specification.

Since going faster entails more signal integrity (SI) issues, PCIe 5 adds more capabilities under the hood for handling signal loss and noise than were built into PCIe 4.

PCIe 5 vs. PCIe 4

Here’s a handy by-the-numbers comparison of PCIe 5 vs. PCIe 4 with the actual aggregate bandwidth adjusted for the encoding overhead.

Comparison table: PCI express 5 vs PCIe 4
Comparison table: PCIe 5 vs PCIe 4

 

PCIe 5: Applications & Market Adoption

AI/ML and Cloud Computing

No surprise, PCIe 5 is the fastest PCI Express ever. While the speed upgrade makes the applications of today run faster, what’s particularly exciting is that PCIe 5 is enabling new applications in markets such as AI/ML and cloud computing.

AI applications generate, move and process massive amounts of data at real-time speeds. An example is a smart car which can generate as much as 4TB of data per day!

But that’s not all, the size of AI/ML training models are doubling every 3-4 months. The torrent of data, and the rapid growth in training models is putting tremendous stress on every aspect of the compute architecture, with interconnections between devices and systems being of critical importance. Also critical is fast access to memory as AI/ML workloads are extremely compute intensive.

But while AI/ML is one major megatrend, there are others. Data centers are changing, with enterprise workloads moving to the cloud at a rapid pace. Those applications mean moving more data, often with real-time speed and latency.

This shift to the cloud, along with ever-more sophisticated AI/ML applications, is accelerating the adoption of higher speed networking protocols that are experiencing a doubling in speed about every two years: 100GbE ->200GbE-> 400GbE->800GbE.

Now this is where PCI express 5 comes in. PCIe 5 delivers an aggregate link bandwidth of almost 128GBps in a x16 configuration. Put simply, PCI express 5 effectively addresses the demands of AI/ML and cloud computing by supporting higher speed networking protocols as well as higher speed interconnections between system devices..

Complete PCI Express 5 Interface Solutions from Rambus

Now that we’ve explained the important benefits of PCI express 5, where can you get a reliable PCIe 5 interface?

Rambus offers a comprehensive interface solution for PCI Express 5 (PCIe 5.0) that consists of a PCIe 5.0 PHY and a co-verified PCIe 5.0 digital controller (from recently acquired Northwest Logic).

Want to know the best part? Having a comprehensive solution with a co-verified PHY and controller means that integration complexity is significantly reduced for chip designers.

What if you already have a PCIe 5 PHY or controller from another vendor or have developed one in house?

Both the Rambus PCIe 5 PHY and controller can be paired with PIPE 5.2 – compliant 3rd-party solutions if so desired. In addition, both PHY and controller are backward compatible to PCIe 4, 3 and 2.

But that’s not all. The Rambus PCIe 5 PHY supports the Compute Express Link (CXL) connectivity standard, a new high-speed interconnect between CPUs and workload accelerators or other attached devices. This helps support accelerators used to complement CPUs in AI/ML applications.

Conclusion

In “PCI Express 5 vs. 4: What’s New?” we explain how PCI Express is the system backbone that transfers data at high bandwidth between CPUs, GPUs, FPGAs and ASIC accelerators using links of variable lane widths depending on the bandwidth needs of the linked devices.

We also detail how the latest PCI Express standard, PCIe 5, represents a doubling over PCIe 4 with a raw speed of 32GT/s vs. 16GT/s translating to aggregate bandwidth for a x16 link of ~128GB/s vs. ~64GB/s.

We then explored how the higher data rates of PCIe 5 are enabling system designers to support a new generation of cloud computing and AI/ML applications.

Lastly, we highlighted Rambus’ comprehensive interface solution for PCI Express 5 (PCIe 5.0) which comprises a PCIe 5.0 PHY and a co-verified digital controller. This comprehensive solution significantly reduces integration complexity for chip designers while providing state-of-the-art performance.

 

California’s IoT law is a good start, but more needs to be done

by Leave a Comment

Written by Paul Karazuba, head of product, Rambus Security

Passed by former California governor Jerry Brown, cybersecurity law SB-327 is slated to go into effect on January 1, 2020. This proactive legislation requires manufacturers to equip IoT devices with “reasonable” security features to prevent unauthorized access, modification and data leaks. Specifically, SB-327 requires manufacturers to implement a unique preprogrammed (default) password for each device. Additionally, manufacturers must ensure that users create a new password the first time a device is activated. Together, these steps are expected to help protect California consumers, as hackers are known to routinely target vulnerable devices shipped with generic or default login credentials.

From our perspective, SB-327 is clearly long overdue. Indeed, unprotected IoT devices continue to pose a threat to both consumer privacy and security across the country. For example, a Ring camera installed in the Memphis bedroom of a young girl was recently hijacked by a hacker who seized control of the device to spy on the 8-year-old, taunt her with music and encourage destructive behavior. Another instance of a Ring camera falling victim to a hacker was reported in December by a Houston family who heard an eerily disembodied voice ask if “anyone [was] home” and promised it was “gonna find out.”

According to various reports, the recent spate of Ring hacks likely involved basic attack techniques such as credential stuffing. This simple process involves accessing accounts with stolen account credentials and large-scale automated login requests. Consequently, Ring users who don’t enable the optional two-step authentication skip setting a unique password or recycle credentials across multiple online services, and are at a greater risk of being hacked. To be sure, malicious hackers have coded dedicated software for breaking into Ring security cameras. Beyond Ring cameras, a wide range of vulnerable consumer IoT devices are frequently targeted by hackers who actively search for devices with default or weak login credentials such as “admin” usernames and “1234” passwords.

Although SB-327 sets an important precedent by requiring a unique preprogrammed (default) password for each IoT device, we believe much more needs to be done to secure connected devices. Security starts at the hardware level, and it should begin on day one of product design. Device designers need to prioritize security as a primary design goal of a connected device; not an afterthought, and certainly not lip service. A solid start to security is basing the foundation of your security in silicon; specifically, a siloed security co-processor capable of executing all security-centric processes completely independently of the main CPU.

Our CryptoManager Root of Trust is an ideal implementation. While located on the same chip as the main CPU, its physical separation and 7 layers of hardware security ensure that secure processes remain exactly that – secure. The root of trust can better help protect consumers by enabling robust remote access authentication and monitoring of anomalous system activity. This siloed approach to security ensures that a potential compromise of the main processor does not expose critical keys and credentials – or impair the execution of security applications that monitor system operation and detect tampering.

Cybersecurity law SB-327 is a good start for California consumers, although far more needs to be done to comprehensively protect IoT devices. Implementing a unique preprogrammed (default) password for each device and requiring users to create a new password can help prevent basic attacks, although a siloed security co-processor is necessary to thwart determined adversaries and complex hacking techniques.

 

Additional Resource:
California’s IoT Law Is A Good Start, But More Needs To Be Done

FREE Webinar: Understanding Fault Injection Attacks and Their Mitigation

Register Today