Ask the Experts: Exploring the Impact of AI 2.0 on the Memory Landscape

In this episode of “Ask the Experts,” Dr. Steven Woo from Rambus Labs discusses the evolution of Artificial Intelligence (AI) from AI 1.0 to AI 2.0. As Dr. Woo explains, AI 2.0 is generative AI that can interpret complex requests and generate a wide range of outputs, including text, speech, code, images, music, video, and 3D models.

AI 2.0 has been enabled by large language models (LLMs) and the ability to generate new content using AI techniques. Woo also detailed the four-step process of training LLMs, which includes preparing a training set, deciding on the model’s architecture, iterative training, and fine-tuning.

The discussion also touched on the democratization of AI, with Woo predicting that the semiconductor industry’s advancements will enable more local inference and training. Lastly, Woo highlighted the importance of chip technologies in AI’s future, emphasizing the need for faster, more efficient processors and technologies for moving and securing data.

Expert

• Dr. Steven Woo, Fellow and Distinguished Inventor, Rambus Labs

Key Takeaways

1. AI 2.0 Advancements: AI 2.0 is the next-generation of AI: generative AI, meaning it can interpret complex requests and produce a wide range of multi-modal outputs such as text, speech, code, images, video, and even 3D models. This advancement is enabled by the meteoric growth in LLM size and resultant capabilities.
2. Large Language Model Training: Training large language models involves four steps: preparing a good training set, deciding on the model’s architecture, iterative training to improve accuracy, and fine-tuning the model to maintain accuracy and effectiveness.
3. AI Democratization: The democratization of AI is currently happening on the inference side, with LLMs being open-sourced and made accessible to millions of people. Over time, as semiconductors become more advanced and affordable, local training will also become democratized, allowing devices like cell phones and laptops to tune themselves to individual users’ needs.
4. AI Chip Technologies: Key chip technologies for AI 2.0 will need to provide greater performance at better power efficiency. This includes faster and more capable processing cores, technologies for moving and securing vast amounts of data, and techniques for reliable data transmission at higher data rates.
5. AI 2.0 Challenges: Challenges in AI 2.0 include managing the power associated with moving data, dealing with on-die errors and maintaining reliability in memory cells across process shrinks, and ensuring reliable data transmission at higher data rates. The industry is actively researching these areas to continue providing the performance levels needed for the rapid developments in AI 2.0.

Key Quote

Now we’re entering the era of AI 2.0, generative AI, and what that means is we’re able to take inputs like speech or text and form very complex requests that these generative AI engines will take and produce multi-modal outputs. They can produce text as an output, they could produce speech, they could generate code, images, music, video and even 3D models. So, the range of content these models can produce is greatly advanced and an extension over what AI 1.0 was able to do.

Related Content

• Webinar: AI 2.0 Driving Demand for Memory Performance
• Whitepaper: Supercharging AI Inference with GDDR7
• Product: Memory Interface Chips
• Product: Interface IP
• Product: Security IP