General

Training AI/ML algorithms

A new survey commissioned by Alegion  reveals that 8 out of 10 companies have found training AI/ML algorithms more challenging than originally anticipated. Moreover, 78 percent of AI/ML projects have stalled at some stage before deployment. 96 percent of enterprises have also encountered data quality and labeling challenges – while only half of enterprises have released an AI/ML project into production.

In addition, only 65 percent of projects have reached the data labeling and algorithm training phase, while 72 percent of those surveyed report that production-level model confidence will require more than 100,000 labeled data items. Nevertheless, 74 percent are using AI and ML projects to disrupt the marketplace, while 82 percent are satisfied with the progress of their respective projects.

“The single largest obstacle to implementing machine learning models into production is the volume and quality of the training data,” states Nathaniel Gates, CEO and co-founder of Alegion. “This research reinforces our own experience, that data science teams new to building ROI-driven systems try to tackle training data preparation in house and get overwhelmed.”

Gates also notes large businesses with more than 100,000 employees are most likely to have an AI strategy – but only 50 percent of them currently have one. Alegion’s survey, says Gates, confirms previous research that AI is still nascent in the enterprise.

AI spending to grow nearly $35.8 billion in 2019

Despite the evolutionary growing pains described above, IDC forecasts that worldwide spending on AI systemswill reach $35.8 billion in 2019, representing an increase of 44 percent over the amount spent in 2018. This number is expected to more than double to $79.2 billion in 2022 with a compound annual growth rate (CAGR) of 38 percent.

More specifically, IDC analysts projects that global spending on AI systems will be led by the retail industry, with companies investing $5.9 billion in 2019 on items such as automated customer service agents, as well as expert shopping advisors and product recommendations. Meanwhile, the banking sector is expected to allocate $5.6 billion for AI-enabled solutions such as automated threat intelligence and prevention systems, as well as fraud analysis and investigation systems. Discrete manufacturing, healthcare providers and process manufacturing will complete the top 5 industries for AI systems spending in 2019.

AI/ML to take center stage at Hot Chips

Perhaps not surprisingly, AI/ML will be taking center stage at Hot Chips 2019, with Rick Merritt of the EE Timesreportingthat half of the talks at the 2019 summer conference are slated to focus on AI acceleration. The annual gathering for microprocessor designers, says Merrit, traditionally focused most of its talks on CPUs for PCs and servers.

“If Rip Van Winkle fell asleep in 1999 and woke up now, he would be astounded by all the attention to machine learning and AI, which were pretty much just research topics when he started his nap,” veteran microprocessor analyst Nathan Brookwood of Insight64 tells the EE Times. “[But Rip would] be pretty comfortable with about half of the papers on this year’s Hot Chips agenda because they are fairly straightforward extrapolations of past conferences. Intel, AMD and IBM are still slugging it out to get more performance out of architectures [that] Rip already knew.”

Researchers at MIT have created a way to determine robustness levels of machine learning (ML) models for various tasks.  And that is done by detecting when those models make mistakes they’re not supposed to make

.

Rob Matheson of the MIT News Offices writes about that development in ECN.

A good part of Matheson’s piece gives the reader some basic tutorial about the new lingo AI and machine learning present to the average person.  For example, he describes convolutional neural networks (CNNs) as “designed to process and classify images for computer vision and many other tasks.”

But tiny things the human eye cannot see can cause a CNN to trigger a completely different classification or modification, according to Matheson.    Those modifications are so-called “adversarial examples.”  That’s what MIT researchers zero in on.  By studying the effects of adversarial examples on neural networks, researchers determine how their models could be vulnerable to unexpected inputs in the real world.

Matheson explains that CNNs process images through many computational layers containing units called neurons. For CNNs that classify images, the final layer consists of one neuron for each category. The CNN classifies an image based on the neuron with the highest output value.

But he adds, “Consider a CNN designed to classify images into two categories: “cat” or “dog.” If it processes an image of a cat, the value for the “cat” classification neuron should be higher. An adversarial example occurs when a tiny modification to that image causes the “dog” classification neuron’s value to be higher.”

The researchers’ technique checks all possible modifications to each pixel of the image. Basically, if the CNN assigns the correct classification (“cat”) to each modified image, no adversarial examples exist for that image.

As part of this article, Matheson reports that behind the technique is a modified version of “mixed-integer programming,” an optimization method where some of the variables are restricted to be integers. Essentially, mixed-integer programming is used to find a maximum of some objective function, given certain constraints on the variables and can be designed to scale efficiently to evaluating the robustness of complex neural networks.

One MIT researcher is quoted in the article as saying, “Across different networks designed for different tasks, it’s important for CNNs to be robust against adversarial examples. The larger the fraction of test samples where we can prove that no adversarial example exists, the better the network should perform when exposed to perturbed inputs.”

These days, there’s considerable talk and hoopla surrounding artificial intelligence/AI.  Tech companies on a worldwide basis are talking about how their products are complying with AI requirements.

And that includes Rambus with its lineup of new GDDR6 and HBM2 PHYs. These products provide SoC and system designers the right solutions to move onward with next generation AI, data center, ADAS, and 5G projects, among others with high speed data rates.

Meanwhile, one legal expert is casting a modicum of doubt. This is making some AI leaders to stop and scratch their heads and wonder what it means to them and their products.

Regulate AI?

The question being posed is this: “Is it too early to start thinking about regulating artificial intelligence?”

You ask, what regulations?  What is there to regulate about AI?

Those questions and associated arguments are reported in security editor Warwick Ashford’s article in ComputerWeekly.com.  Ashford centers his piece on an earlier talk that Dr. Karsten Kinast made at the European Identify & Cloud Conference 2019.   Dr. Kinast is described as a legal expert and fellow analyst at KuppingerCole, an international analyst organization.

Referring to AI in a universal sense, Dr. Kinast prefaced his argument for AI regulations by saying that regulation needs to be introduced as soon as possible in order to ensure that AI brings maximum benefit to society.

It is the strong belief of Dr. Kinast that it’s important to assure positive AI goals are reached.  But then he also cautions that “no single technology company or small set of tech companies is allowed to dominate setting the rules and the agenda for AI.”

Ashford writes that inroads are being made along those lines.  He reports that in early April 2019, the European Commission (EC) published a set of principles around the ethical application of AI.  Plus, it announced a major pilot in which the guidelines will be applied.

Dr. Kinast argued that these EC actions are not enough, but that more must be done on a global scale. “Once truly self-learning AI is a reality, it will be too late to start trying to regulate it.

As an added precaution, he said that “we need to guard against AI regulating us in the same way technology is already regulating how we communicate, how we think, how we look for solutions to problems and how we interact with others.

EE Times reports that Graphcore of Bristol, UK has put on the market a new type of processor for AI acceleration.  It’s called the intelligence processing unit (IPU). According to CEO Nigel Toon, the IPU processor, is the most complex processor chip that’s ever been built.

It’s described as “just shy of 24 billion transistors on a single die, in 16 nm. Each chip delivers 125 teraFLOPS, and we can put eight of those cards into a standard 4U chassis and connect them together through IPU links.”

Toon went on to tell EE Times reporter, Sally Ward-Foxton, that the processors can work together as a single processing element that delivers two petaFLOPS of compute, but in a different form to what exists in CPUs and GPUs, which provides a much more efficient processing platform for machine intelligence. These modules will go into servers for cloud computing, and potentially into autonomous vehicles as well.

Ward-Foxton asked Toon about the IPU’s performance compared to leading GPUs on the market.  His response was, “If you’re doing feed-forward convolutional neural networks used for classification of static images, GPUs do that quite well. We would be able to offer a performance advantage of two or three, sometimes five times.

With much more complex models, those that have data passing through and then feeding back to try and understand context (conversations, for example), you’re passing the data through a number of times and you need to do that very quickly. Because all of the model is held inside our processor, on applications like that, we’re much faster than a GPU, maybe ten, twenty or fifty times faster.”

As for IPU’s suitability for inference and training, Toon’s responded saying, “Yes, you can use the same IPU chip for inference as well as training. That was very important to us, from an architectural point of view, since as machine learning evolves, systems will be able to learn from experience.”

Toon explained that the keys to inference performance are as follows: low latency and being able to work with small models, small batches, and trained models where you might be trying to introduce sparsity into the model.

He said, “We can do all these things efficiently on the IPU. So, in that 4U chassis, where you’ve got 16 IPUs all working together to do training, we could have each of those IPUs running a separate inference task, controlled by a virtual machine running on a CPU. What you end up with is a piece of hardware that you can use for training. Then, once you’ve trained the models, deploy it, but then as the models evolve and we start to want to learn from experience, the same hardware can be used to do that.”

From a technology perspective, all sounds good. However, what about the competition and especially GPUs already on the market?  Toon said that GPUs are holding back new innovations.

He concluded by saying, “If you look at the types of models that people are working on, they are primarily working on forms of convolutional neural networks because recurrent neural networks and other kinds of structures, [such as] reinforcement learning, don’t map well to GPUs. Areas of research are being held back because there isn’t a good enough hardware platform, and that’s why we’re trying to bring IPUs to market.”

by Steven Woo

Neural networks (NNs) span a wide range of topologies and sizes. Some neural networks are relatively simple and have only two or three layers of neurons, while so-called deep neural networks may comprise 100+ layers of neurons. In addition, the layers can be extremely wide – with hundreds to thousands of neurons – or they can be much narrower with as few as a half dozen neurons, for example. Determining the topology and size of a neural network for a specific task is typically based on a combination of experimentation and comparison to similar solutions.

Training, Not Programming

Instead of being programmed, neural networks are trained to perform certain tasks like classification and identification. The training phase involves iteratively providing examples – or training sets – to the neural network and presenting the desired outcome. In some cases, the desired outcome is determined a priori by labeling the data. For example, a neural network might be trained to identify cats and dogs from a series of images. In this case, the images in the training set are labeled as having cats or dogs, with the proper type of animal being the desired output.

During training, the characteristics of the neural network, including weights and biases, are iteratively adjusted. This process, known as back propagation or back prop, uses mathematical equations (Stochastic Gradient Descent is one popular method) to increase the likelihood that the same input will produce the desired outcome from the neural network. Once the neural network achieves a high enough accuracy on the training set, it can be deployed in the field to perform inferencing on data it has never seen before.

NN Training Hardware

Depending on application requirements, the training and inference process can be hosted on any number of hardware platforms including graphics processing units (GPUs), custom application-specific integrated circuits (ASICs) and field-programmable gate arrays (FPGAs). Modern training sets can be enormous in size, especially those used for image classification in data center environments. Until recently, the hardware of choice for training neural networks has been graphics adapters due to their combination of performance, power efficiency and high memory capacity to hold large training sets. More recently, a resurgence of interest and funding for specialized neural network silicon has resulted in a number of purpose-built neural network chips becoming available that achieve even higher levels of performance and power-efficiency, making them the new solution of choice for neural networks.

The performance of hardware designed for neural network training and inference is heavily dependent upon memory bandwidth. The memory system typically holds the neural network parameters – weights and biases – along with training data. With hardware optimized for performing fast computations, neural network hardware continually stresses the memory system as model parameters as well as training and inference data are fetched from memory. In the case of training, the process of cycling through the training data as quickly as possible strains both memory capacity and bandwidth. For this reason, the training sets and model parameters are held in local memory to avoid data transfers over the much slower PCIe interconnects.

Neural Networks: Deployment and Future Evolution

While still at a relatively nascent stage, the current wave of AI innovation has seen numerous advances in both hardware and software, with a robust emphasis on co-design. A significant percentage of modern teams are composed of software staff, in some cases outnumbering their hardware engineering counterparts. This paradigm supports a diverse range of AI applications and addresses a key industry demand for hardware that is designed to run specific software and vice versa.

Co-design efforts have yielded interesting architecture and software innovations including reduced-precision computation, new topologies and new training algorithms that take advantage of mixed-precision to strike a balance between training speed, accuracy and power consumption. We will be taking a closer look at reduced-precision computation techniques in future blog posts.

Conclusion

Instead of being programmed, neural networks are trained to perform certain tasks such as classification and identification. The training phase involves iteratively providing examples – or training sets – to the neural network and presenting the desired outcome. Depending on application requirements, the training and inference process can be hosted on any number of hardware platforms including graphics processing units (GPUs), custom application-specific integrated circuits (ASICs) and field-programmable gate arrays (FPGAs). More recently, specialized neural network silicon has been introduced into the market and have become the new solution of choice.

Interested in reading more about machine learning and neural networks? You can browse our article archive on the subject here.