Rambus

At Rambus, we create cutting-edge semiconductor and IP products, spanning memory and interfaces to security, smart sensors and lighting.

Home > Blogs

Autonomous Vehicles: Memory Requirements & Deep Neural Net Limitations

by Leave a Comment

Written by Steven Woo

Introduction

According to the National Highway Traffic Safety Administration (NHTSA), various driver assistance technologies are already helping to save lives and prevent injuries. Specific examples include helping drivers avoid making unsafe lane changes, warning of other vehicles when backing up, or automatically braking when a vehicle ahead stops or slows abruptly. The above-mentioned safety technology leverages a wide range of hardware – including sensors, cameras and radar – to help vehicles identify safety risks and warn drivers to avoid crashes and collisions.

Self-Driving Memory Requirements

However, many of these systems are using DRAM-based memory solutions with relatively modest bandwidths. This is because most advanced driver-assistance systems (ADAS) are rated Level 2 and only offer drivers partial automation capabilities. Future self-driving cars (Level 3 – Level 5) will require new generations of memory with significantly increased bandwidths. This additional bandwidth will enable autonomous vehicles to rapidly execute massive calculations and safely implement real-time decisions on roads and highways, with the ultimate goal of doing this without any driver assistance.

Automotive systems in 2020 models are slated to be equipped with x32 LPDRAM components at I/O signaling speeds up to 4266Mb/s. However, Micron estimates that advanced driver assistance systems (ADAS) applications will demand 512GB/s – 1024 GB/s bandwidth to support Level 3 and 4 autonomous driving capabilities. According to Micron, GDDR6, with a data transfer rate of 16 Gbps, is a “fundamental technology” that provides the essential memory bandwidth to fuel on-board AI compute engines (deep neural nets) and drive autonomous vehicles. At Level 5, or full autonomy, vehicles will be capable of leveraging sophisticated machine learning algorithms to independently react to a dynamic environment by fully recognizing traffic signs and stoplights, as well as accurately predicting the actions of cars, trucks and pedestrians.

It should be noted that a number of companies are already designing and marketing dedicated silicon for the nascent self-driving market. Nvidia, for example, launched its AGX self-driving compute platforms in 2018. These platforms are built around the company’s Xavier SoC, a processor specifically made for autonomous driving. The SoC incorporates six different types of processors to run redundant and diverse algorithms for AI, sensor processing, mapping and driving. With Xavier, DRIVE AGX platforms can process data from camera, lidar, radar and ultrasonic sensors to comprehend and navigate a complex 360-degree environment in real-time.

Deep Neural Networks & The Limits of Automotive Autonomy

However important, hardware represents only one side of the autonomous vehicle equation. It is also critical to understand the current limitations of deep neural networks (DNNs) and deep learning models as the self-driving space evolves. For example, a recently published academic paper titled “DeepTest: Automated Testing of Deep-Neural-Network-Driven Autonomous Cars” notes that DNNs have successfully enabled the development of safety-critical machine learning (ML) systems for autonomous vehicles. However, the paper warns that DNNs are still susceptible to incorrect and unexpected cornercase behaviors that have led to collisions. As the paper’s authors point out, existing mechanisms for detecting erroneous behavior are heavily contingent upon the manual collection of labeled test data or ad hoc, unguided simulation. Because autonomous vehicles adapt their behavior based on sensor input, the space of possible inputs is quite significant.

As such, it is unlikely that unguided simulations alone would be robust enough to positively identify the full range of erroneous behaviors involved in autonomous navigation. DeepTest – a potential solution proposed in the paper – addresses this limitation by maximizing the neuron coverage of a DNN using synthetic test images that are generated by applying different realistic transformations on a set of seed images. Put succinctly, this approach uses domain-specific metamorphic relations to identify erroneous behaviors of the DNN without detailed specification.

In addition to the above-mentioned issue surrounding cornercase behaviors, another recent paper discusses the concept of deliberate physical-world attacks against deep learning models. Specifically, researchers managed to fool two road sign classifiers by using a perturbation in the shape of black and white stickers to attack a stop sign. This caused targeted misclassification in 100% of the images obtained in controlled lab settings and above 84% of the captured video frames obtained on a moving vehicle.

Conclusion

As the NHTSA states, the dream of fully autonomous cars and trucks that drive us instead of us driving them will one day be a reality. Although the self-driving space is still very much a work in progress, a number of companies are designing a new generation of dedicated silicon that require high bandwidth capabilities. Concurrently, deep neural networks and deep learning models continue to evolve as researchers grapple with and address various algorithmic limitations and vulnerabilities.

Taking the Show on the Road

by Leave a Comment

Starting later this month, Rambus will host a series of lunch & learns around the globe entitled “Securing Electronic Systems at their Foundation.”  These technical sessions are designed for engineering, product, and program management staff at both defense and commercial OEMs. Interactive in nature, they will address specific security issues facing attendees. Topics that attendees will have an opportunity to learn about and discuss include:

  • Architectural options for implementing security in electronic devices, from software to hardware, with focus on a range of hardware security approaches
  • Selecting the right security IP, navigating regulations and certifications, and implementing it in your product
  • Integrating security hardware into a secure supply chain

We begin the tour in two locations in Southern California in late July, then continue to Denver, Orlando, Boston, and then travel across the pond to Herzliya, Israel, and finish up in Yokohama, Japan. Sessions will feature talks from a number of Rambus security experts, as well as guest talks from senior technical staff of Tortuga Logic at specific events. More information on the sessions, including registration information and speaker bios, can be found here.

If we’re not coming to your town, but you still want to hear more? These lunch & learns build on the topics presented in our recent three-part Secure Silicon IP webinar series, available on demand. Additionally, we are presenting and demonstrating our solutions during the ongoing SiFive RISC-V technical symposiums.

Let’s talk about neuromorphic computing

by Leave a Comment

The term ‘neuromorphic computing’ can be traced back to the 1980s when Caltech researcher Carver Mead first proposed the concept of designing integrated circuits (ICs) to mimic the organization of living neuron cells. As the National Institute of Standards (NIST) states, neuromorphic computing promises to dramatically improve the efficiency of computational tasks such as perception and decision making.

Nanoscale oscillators and reconfigurable Josephson junctions

Although software and specialized hardware implementations of neural networks have made significant progress in recent years, such implementations are still are still ‘many orders of magnitude’ less energy efficient than the human brain. This is precisely why NIST researchers are working on several bio-inspired hardware implementations of neuromorphic networks. More specifically, one project is based on high-frequency room-temperature nanoscale oscillators utilizing the spin-torque effect, while the other is designed around dynamically reconfigurable magnetic Josephson junctions operating at liquid-helium temperatures.

Neuromorphic silicon: Loihi and Akida

On the silicon side, Intel introduced a self-learning neuromorphic chip dubbed ‘Loihi.’ The chip mimics
how the human brain functions by learning to operate based on various modes of feedback from its environment. The energy-efficient chip – which uses data to learn and make inferences – becomes smarter over time, without the need for traditional training methodologies. Rather, Loihi takes a novel approach to computing via asynchronous spiking.

Similarly, BrainChip developed the Akida NSoC, which the company describes as a neuromorphic chip engineered on a digital logic process. Inspired by the biological function of neurons, these spiking neural networks (SNNs) are inherently lower power than traditional convolutional neural networks (CNNs). Indeed, SNNs replace the math-intensive convolutions and back-propagation training methods of CNNs with neuron functions and feed-forward training methodologies.

IBM’s True North neuromorphic chip

A patent filed by IBM in late 2018 offers a rare insight into the company’s True North neuromorphic chip. According to Nicole Hemsoth of The Next Platform, one of the elements making IBM’s neuromorphic device more efficient and scalable is a new algorithm that determines placement, or how a neurosynaptic core in True North is mapped onto the chip (and even across chips). In addition, the patent details how the new device supports standard deep learning frameworks that can be used and converted during offline training. As Hemsoth explains, the trained networks are converted into corelets after training and subsequently converted into model files that can be loaded onto specific neurosynaptic substrates.

Nengo and algorithms

On the software side, Applied Brain Research developed Nengo, an advanced tool that allows programmers to more easily run AI algorithms on new cortical chips. According to Wired, the Python-based compiler is targeted at AI applications that will run on general purpose neuromorphic hardware. Nengo has already been used to develop vision systems, speech systems, motion control and adaptive robotic controllers.

Memory for neuromorphic systems

As a recent article in Advanced Material Technologies emphasizes, artificial neurons should closely mimic biological neurons, while artificial synapses should accurately emulate biological synapses. Both artificial implementations must also be power‐efficient, scalable and capable of implementing relevant learning rules to facilitate large‐scale neuromorphic functions. Thus far, a wide range of memory devices have been utilized to realize the goal of creating efficient artificial synapses. These include resistive random‐access memory (ReRAM), diffusive memristors, phase change memory (PCM), ferroelectric field‐effect transistors (FeFET), spintronics-based MRAM and synaptic transistors.

“Each of these devices has its own strengths and weaknesses. One type of device can be preferred over the others depending on the requirements for a specific application,” the article states. “Although attempts have been made to implement logic circuits using emerging devices, however, at least in the near future, neuromorphic systems still need the mature and reliable CMOS circuitry to implement peripheral components, which means neuromorphic systems and CMOS circuits will complement rather than replace each other.”

Interested in learning more about various types of memory? You can browse our article archive on the subject here.

Securing the Infrastructure with the Global Semiconductor Alliance

by Leave a Comment

By Paul Karazuba, Senior Director of Product Marketing, Cryptography

Security, as always, was a hot topic at the recent Global Semiconductor Alliance (GSA) Silicon Summit in Santa Clara, CA. As the recognized leader in semiconductor and device security, Rambus played a large part in the summit. As part of our participation, Neeraj Paliwal, VP of Products at Rambus Cryptography, delivered a speech titled “Securing the Infrastructure” during the Smart Connected Industries and Infrastructure portion of the summit.

Neeraj Paliwal at GSA

During the talk, Neeraj explained that for any smart connected device, security must start at the silicon level. Further, it’s not just making sure your chip has some amount of security, but rather that chip and device OEMs need to make sure their chips are designed, manufactured, and deployed in-field with security in mind for the Infrastructure to truly be secure.

Neeraj started with detailing securing silicon at the design phase and that the race for processor speed coupled with the economics of Moore’s Law has led to a situation where modern processors may not meet the security requirements demanded by consumers. The myriad of processor security breaches of late supports this. Rambus advocates a siloed hardware root of trust co-processor approach to chip security. This approach allows the main CPU to do what it does best – run extremely fast and efficiently – while allowing the root of trust to form a trust anchor for secure apps and processes within a system.

However, as Neeraj pointed out, you need to do more than this. In order for a chip to be truly secure, that chip needs to be manufactured with security in mind. To Rambus, this involves the inclusion of robust device provisioning systems into chip manufacturing, regardless of whether that manufacturing is done in captive (trusted) or 3rd party (untrusted) facilities. Automated systems allow each chip to be given a unique and immutable identity that is cryptographically bound to the chip, stored in the root of trust. Without provisioning, it is extremely difficult, if not impossible, to guarantee the identity and provenance of the chip.

Finally, Neeraj spoke of the importance of trusted, in-field cloud services. As he put simply, “If you can’t trust data coming from a device, you can’t trust the device itself.” He explained that by using the information provisioned into the device during manufacturing, the device itself can be authenticated and attested. Having this capability allows the device OEM a whole host of product security features, including functions like secure boot, secure FOTA (firmware over the air updates), and others.

Rambus believes that the most holistic and effective approach to securing our connected devices starts with securing the silicon inside, and building outwards from that trust anchor.

Visa to Acquire Rambus Payments and Ticketing Businesses

by Leave a Comment

We are pleased to share that Visa has signed a definitive agreement to acquire our Rambus Payments and Ticketing businesses. This is exciting news as Visa is one of the industry leaders in tokenization technology which replaces sensitive payment information with a unique identifier, or “token”, to make digital payments safer. Rambus payment token technology will enable Visa to extend the security and convenience of tokenization to all types of transactions beyond Visa cards, including those on domestic card networks, account-based and real-time payments systems.

In addition, the digital ticketing portfolio and expertise from Rambus will complement Visa’s efforts in transport. Visa is committed to delivering transit and urban mobility solutions to public transit operators, technology partners and cities around the world, with nearly 250 transit projects underway, helping millions of people get where they are going faster and easier than ever before.

TS Anil, SVP, global head of payment products and platforms, Visa commented: “Facilitating safer, more secure digital transactions is core to Visa’s brand promise and central to growing electronic payments for everyone, everywhere. As the way people and businesses pay and get paid continues to evolve, the addition of Rambus’ technology will allow us to deliver greater security beyond the card to support more transactions, payments systems and participants. Going forward, we will apply these expanded capabilities, expertise and scale to help further all forms of global commerce.”

Jerome Nadel, SVP/GM, payments and ticketing and CMO, Rambus said: “Rambus Payments and Ticketing solutions are at the forefront of tokenization technology. Joining forces with Visa as a payments leader will allow the group to scale technology and capabilities to deliver new products and services to the market, while continuing to partner with existing customers.”

View the press release here.

When a Chip Just Isn’t a Chip

by Leave a Comment

When most people imagine counterfeit goods, they tend to picture the ‘Rolax’ watch that you can buy from that somewhat shady guy behind the local watering hole, or the knock-off purse your relative brought you back from vacation. Most don’t imagine their new security camera containing non-authentic components, or that the military plane seen on the news might be flying with counterfeit chips. Scary, but it’s a reality.

Counterfeit semiconductors are everywhere. Industry estimates are up to 5% of military and medical equipment contain counterfeit parts. The issue isn’t unique to any particular application or geography. In 2017’s “Operation Wafer,” the European-wide Joint Customs Operation (JCO) seized more than one million counterfeit semiconductor devices during a 2-week operation. One million devices – in just two weeks! Industry Week has pegged the fake semiconductor market at $75B, with Havocscope reporting more than $169B in counterfeit parts circulating in the marketplace. The problem is so prevalent that the Global Semiconductor Alliance started a working group on supply chain security.

So…why should fake chips matter to you? Lets talk safety. There is no way to understand how counterfeit parts function. Are they actually doing what the original (authentic) part is supposed to do, or are they operating differently? An even scarier thought, are they intentionally compromising the systems around them? Or are they passing information they gather to an adversary? Confirmed recent incidents of counterfeit parts being found in the field include automated external defibrillators (AED), airport landing lights, intravenous (IV) drip machines, and braking systems for high speed trains. Each of these represent a significant risk to human health and safety.

Device OEMs are forced to address a key question, “if we can’t trust the authenticity of semiconductor components we buy, how can we (and our customer) really trust the devices we make?” Frankly, the answer is “we can’t.”

So how can we fix this? Trust starts at the silicon level, but that trust is only as good as the security applied during manufacturing. That’s where the Rambus CryptoManager Infrastructure becomes a highly valuable tool towards guaranteeing semiconductor authenticity, starting at time of initial manufacturing and stretching all the way to end of life.

During the manufacturing of a chip, whether at an OEM or 3rd-party facility, CryptoManager Infrastructure securely provisions (injects) each and every semiconductor with a unique cryptographic key, or other secure data, in a known-secure area of the chip. Each key is unique to the individual chip and forms the basis of a trusted identity. The process is completely automated. There is no human intervention, allowing the process to take place in just about any facility around the world. Keys are securely generated in air-gapped systems, and only known to the OEM. Once the chip leaves the factory and is placed into a device, the authenticity of that chip can be checked at any time using the Rambus Key Management Service (KMS).

Chip OEMs who use our infrastructure product can provide a chip authenticity guarantee to their device OEM customers, who can then provide the same guarantee to their customers. By cutting down the number of counterfeit chips, we lower the risks to safety and security in electronic devices.