General

A recent paper published on arXiv by a team of UC Berkeley researchers observes that neural networks are increasingly bottlenecked and constrained by the limited capacity of on-device GPU memory. Indeed, deep learning is constantly testing the limits of memory capacity on neural network accelerators as neural networks train with high-resolution images, 3D point-clouds and long Natural Language Processing (NLP) sequence data.

“In these applications, GPU memory usage is dominated by the intermediate activation tensors needed for backpropagation. The limited availability of high bandwidth on-device memory creates a memory wall that stifles exploration of novel architectures,” the researchers explain. “One of the main challenges when training large neural networks is the limited capacity of high-bandwidth memory on accelerators such as GPUs and TPUs. Critically, the bottleneck for state-of-the-art model development is now memory rather than data and compute availability and we expect this trend to worsen in the near future.”

As the researchers point out, some initiatives to address this bottleneck focus on dropping activations as a strategy to scale to larger neural networks under memory constraints. However, these heuristics assume uniform per-layer costs and are limited to simple architectures with linear graphs. As such, the UC Berkeley team uses off-the-shelf numerical solvers to formulate optimal rematerialization strategies for arbitrary deep neural networks in TensorFlow with non-uniform computation and memory costs. In addition, the UC Berkeley team demonstrates how optimal rematerialization enables larger batch sizes and substantially reduced memory usage – with minimal computational overhead across a range of image classification and semantic segmentation architectures.

“Our approach allows researchers to easily explore larger models, at larger batch sizes, on more complex signals with minimal computation overhead,” says the UC Berkeley team.

Specifically, the paper published by the UC Berkeley team on arXiv describes a formalization of the rematerialization problem as a mixed integer linear program with significantly more flexible search space than prior work. As well, it details an algorithm to translate a feasible solution into a concrete execution plan and a static training graph – and describes an implementation of optimal tensor rematerialization in Tensorflow at runtime. Finally, it introduces Checkmate, a system that enables training models with up to 5.1× larger input sizes and up to 2.6× larger batch sizes (than prior art) at minimal overhead.

Image Credit: Department of EECS, UC Berkeley

“Checkmate enables practitioners to train a high-resolution U-Net on a single V100 GPU with an unprecedented batch size of 57, as well as batch size of 1105 for MobileNet,” the UC Berkeley team states. “The former is 2.6× larger than the maximum batch size with prior art. Checkmate … solves for optimal schedules in reasonable times (under an hour) using off-the-shelf MILP solvers, then uses these schedules to accelerate millions of training iterations.”

The UC Berkeley research team also notes that its method scales to complex, realistic architectures and is hardware-aware via the use of accelerator-specific, profile-based cost models.

“In addition to reducing training cost, Checkmate enables real-world networks to be trained with up to 5.1× larger input sizes,” they conclude.

Commenting on the above, Steven Woo, Rambus fellow and distinguished inventor, states: “The work of the team at UC Berkeley is focused on one of the key problems facing neural networks today – how to keep up with the trend of using larger models and training sets that are growing faster than the memory capacity of individual hardware accelerators can scale. Distributing the training among multiple accelerators is one method, but communication between the accelerators reduces performance and increases the energy used. There is a clear need to do more on each accelerator if possible.”

In late 2018, six researchers published a DEF CON report that detailed a range of critical security vulnerabilities affecting voting equipment and related computer systems. These include supply chain insecurity, proven susceptibility to remote attacks, machines that can be hacked faster than an actual vote and perpetually unpatched vulnerabilities. Let’s take a closer look at these shortcomings and vulnerabilities below.

Supply Chain Insecurity, Remote Attacks & Unpatched Hacks

According to the DEF CON report, the global voting machine parts supply chain lacks consistent security procedures capable of verifying whether components are trustworthy or pre-hacked before a machine is assembled. In practical terms, this means compromised chips could allow an attacker to remotely hack entire classes of machines across the United States.

In addition, the report confirms that voting machines are not effectively air gapped from the internet and can be remotely hacked. Moreover, machines in at least 15 states can be physically hacked with a pen in two minutes. This means an attacker could potentially compromise a machine while ‘voting’ in a polling place on Election Day. Perhaps most disturbingly, voting machine security flaws frequently go unpatched and unfixed by manufacturers.

Senators Demand Answers from Election Equipment Manufacturers

The real-world risks of deploying voting machines plagued by unpatched security vulnerabilities recently raised the ire of four senior senators who questioned election equipment makers as to why they continued to sell the decades-old machines.

“The integrity of our elections remains under serious threat. Our nation’s intelligence agencies continue to raise the alarm that foreign adversaries are actively trying to undermine our system of democracy, and will target the 2020 elections as they did the 2016 and 2018 elections,” the senators wrote in an official letter sent to the election equipment manufacturers. “The integrity of our elections is directly tied to the machines we vote on – the products that you make. Despite shouldering such a massive responsibility, there has been a lack of meaningful innovation in the election vendor industry and our democracy is paying the price.”

Secure Hardware & A Secure Vote: An Industry Mandate

Helena Handschuh, Rambus Fellow and Chair of the RISC-V Foundation’s security standing committee, recently penned an article for the EE Times about the importance making secure hardware an industry mandate. All too often, says Handschuh, companies take shortcuts in design assurance and secure design, trying to meet certain performance, area, speed and cost requirements.

“[However], without secure hardware, design flaws will continue to be discovered and exploited, risking serious data breaches and other serious consequences,” she explains. “It’s time for the industry to move away from the principle of security by obscurity and embrace open architectures like RISC-V which enable anyone to inspect and analyze the instruction set architecture (ISA) to assess its security.”

According to Handschuh, RISC-V Foundation member Galois is a company that actively participates in DARPA’s System Security Integration Through Hardware and Firmware (SSITH) program, developing tools and techniques for quantitatively measuring and reasoning for system security, particularly for hardware. As part of this program, Galois is developing baseline processors from which security improvements will be measured.

“DARPA [also] recently announced that Galois will be developing a voting system as the demonstration vehicle for this secure system, built with fully open source hardware and software,” she writes. “While the voting system is not intended for production, it serves as an important demonstration of how DARPA technology can be used for a critical infrastructure system.”

The voting system, says Handschuh, will be built on open source RISC-V CPUs and incorporate auditable software components, enabling the public to review both the software and the hardware since the RISC-V ISA is public and standardized.

“The purpose of this system is to spur continued research and innovation to develop more secure hardware and software solutions for the benefit of everyone,” she adds. “The voting system will be publicly ‘red teamed’ in the voting village at DEF CON 2019 and DEF CON 2020 so attendees can examine it and conduct penetration tests to gauge its security. The demonstration will include both an end-to-end verifiable and a traditional non-verifiable voting system.”

In 2019, says Handschuh there will be a smart ballot box on SSITH hardware, while the 2020 event will feature all components on SSITH hardware, including both the ballot marking device and optical scan systems. Meanwhile, a scaled down, low cost version of the system will be made available via Crowd Supply so anyone can buy it, experiment with it and use it to run even informal elections such as for school clubs or sports teams. University teams also will be participating in the program to enhance their own security research efforts.

“The future of security is in the hands of developers. We strongly encourage everyone to get involved and work together to tackle the dynamic security demands of this new era of innovation,” Handschuh concludes.

Training AI/ML algorithms

A new survey commissioned by Alegion  reveals that 8 out of 10 companies have found training AI/ML algorithms more challenging than originally anticipated. Moreover, 78 percent of AI/ML projects have stalled at some stage before deployment. 96 percent of enterprises have also encountered data quality and labeling challenges – while only half of enterprises have released an AI/ML project into production.

In addition, only 65 percent of projects have reached the data labeling and algorithm training phase, while 72 percent of those surveyed report that production-level model confidence will require more than 100,000 labeled data items. Nevertheless, 74 percent are using AI and ML projects to disrupt the marketplace, while 82 percent are satisfied with the progress of their respective projects.

“The single largest obstacle to implementing machine learning models into production is the volume and quality of the training data,” states Nathaniel Gates, CEO and co-founder of Alegion. “This research reinforces our own experience, that data science teams new to building ROI-driven systems try to tackle training data preparation in house and get overwhelmed.”

Gates also notes large businesses with more than 100,000 employees are most likely to have an AI strategy – but only 50 percent of them currently have one. Alegion’s survey, says Gates, confirms previous research that AI is still nascent in the enterprise.

AI spending to grow nearly $35.8 billion in 2019

Despite the evolutionary growing pains described above, IDC forecasts that worldwide spending on AI systemswill reach $35.8 billion in 2019, representing an increase of 44 percent over the amount spent in 2018. This number is expected to more than double to $79.2 billion in 2022 with a compound annual growth rate (CAGR) of 38 percent.

More specifically, IDC analysts projects that global spending on AI systems will be led by the retail industry, with companies investing $5.9 billion in 2019 on items such as automated customer service agents, as well as expert shopping advisors and product recommendations. Meanwhile, the banking sector is expected to allocate $5.6 billion for AI-enabled solutions such as automated threat intelligence and prevention systems, as well as fraud analysis and investigation systems. Discrete manufacturing, healthcare providers and process manufacturing will complete the top 5 industries for AI systems spending in 2019.

AI/ML to take center stage at Hot Chips

Perhaps not surprisingly, AI/ML will be taking center stage at Hot Chips 2019, with Rick Merritt of the EE Timesreportingthat half of the talks at the 2019 summer conference are slated to focus on AI acceleration. The annual gathering for microprocessor designers, says Merrit, traditionally focused most of its talks on CPUs for PCs and servers.

“If Rip Van Winkle fell asleep in 1999 and woke up now, he would be astounded by all the attention to machine learning and AI, which were pretty much just research topics when he started his nap,” veteran microprocessor analyst Nathan Brookwood of Insight64 tells the EE Times. “[But Rip would] be pretty comfortable with about half of the papers on this year’s Hot Chips agenda because they are fairly straightforward extrapolations of past conferences. Intel, AMD and IBM are still slugging it out to get more performance out of architectures [that] Rip already knew.”

Researchers at MIT have created a way to determine robustness levels of machine learning (ML) models for various tasks.  And that is done by detecting when those models make mistakes they’re not supposed to make

.

Rob Matheson of the MIT News Offices writes about that development in ECN.

A good part of Matheson’s piece gives the reader some basic tutorial about the new lingo AI and machine learning present to the average person.  For example, he describes convolutional neural networks (CNNs) as “designed to process and classify images for computer vision and many other tasks.”

But tiny things the human eye cannot see can cause a CNN to trigger a completely different classification or modification, according to Matheson.    Those modifications are so-called “adversarial examples.”  That’s what MIT researchers zero in on.  By studying the effects of adversarial examples on neural networks, researchers determine how their models could be vulnerable to unexpected inputs in the real world.

Matheson explains that CNNs process images through many computational layers containing units called neurons. For CNNs that classify images, the final layer consists of one neuron for each category. The CNN classifies an image based on the neuron with the highest output value.

But he adds, “Consider a CNN designed to classify images into two categories: “cat” or “dog.” If it processes an image of a cat, the value for the “cat” classification neuron should be higher. An adversarial example occurs when a tiny modification to that image causes the “dog” classification neuron’s value to be higher.”

The researchers’ technique checks all possible modifications to each pixel of the image. Basically, if the CNN assigns the correct classification (“cat”) to each modified image, no adversarial examples exist for that image.

As part of this article, Matheson reports that behind the technique is a modified version of “mixed-integer programming,” an optimization method where some of the variables are restricted to be integers. Essentially, mixed-integer programming is used to find a maximum of some objective function, given certain constraints on the variables and can be designed to scale efficiently to evaluating the robustness of complex neural networks.

One MIT researcher is quoted in the article as saying, “Across different networks designed for different tasks, it’s important for CNNs to be robust against adversarial examples. The larger the fraction of test samples where we can prove that no adversarial example exists, the better the network should perform when exposed to perturbed inputs.”

These days, there’s considerable talk and hoopla surrounding artificial intelligence/AI.  Tech companies on a worldwide basis are talking about how their products are complying with AI requirements.

And that includes Rambus with its lineup of new GDDR6 and HBM2 PHYs. These products provide SoC and system designers the right solutions to move onward with next generation AI, data center, ADAS, and 5G projects, among others with high speed data rates.

Meanwhile, one legal expert is casting a modicum of doubt. This is making some AI leaders to stop and scratch their heads and wonder what it means to them and their products.

Regulate AI?

The question being posed is this: “Is it too early to start thinking about regulating artificial intelligence?”

You ask, what regulations?  What is there to regulate about AI?

Those questions and associated arguments are reported in security editor Warwick Ashford’s article in ComputerWeekly.com.  Ashford centers his piece on an earlier talk that Dr. Karsten Kinast made at the European Identify & Cloud Conference 2019.   Dr. Kinast is described as a legal expert and fellow analyst at KuppingerCole, an international analyst organization.

Referring to AI in a universal sense, Dr. Kinast prefaced his argument for AI regulations by saying that regulation needs to be introduced as soon as possible in order to ensure that AI brings maximum benefit to society.

It is the strong belief of Dr. Kinast that it’s important to assure positive AI goals are reached.  But then he also cautions that “no single technology company or small set of tech companies is allowed to dominate setting the rules and the agenda for AI.”

Ashford writes that inroads are being made along those lines.  He reports that in early April 2019, the European Commission (EC) published a set of principles around the ethical application of AI.  Plus, it announced a major pilot in which the guidelines will be applied.

Dr. Kinast argued that these EC actions are not enough, but that more must be done on a global scale. “Once truly self-learning AI is a reality, it will be too late to start trying to regulate it.

As an added precaution, he said that “we need to guard against AI regulating us in the same way technology is already regulating how we communicate, how we think, how we look for solutions to problems and how we interact with others.