In recent years we’ve heard plenty about the shortage of chip designer expertise across the board. The answer, the EDA tools companies have been telling us, is generative AI. The keynote given by Aart de Geus at the SNUG event in Santa Clara emphasized this. The presentations at DAC 2023 also reiterated this growing trend for AI in EDA and how it will solve the skills shortage by giving us AI assistants to help existing engineers achieve even more in less time as systems design grows more and more complex.

A research paper presented by Nvidia at the International Conference on Computer-Aided Design this week highlighted how its engineers created for their internal use a custom large language model (LLM), called ChipNeMo, trained on the company’s internal data to generate and optimize software and assist human designers. The idea is that as this research evolves, they could apply generative AI to each stage of chip design and potentially reap significant gains in overall productivity.

Nvidia’s chief scientist, Bill Dally, who presented the LLM at the conference, told Sally Ward-Foxton for EE Times, that ChipNeMo can answer questions about chip design, particularly from junior designers. He said, “It turns out our senior designers spend a lot of time answering questions from junior designers. If the first thing the junior designer can do is to go to ChipNeMo and say, ‘What does this signal coming out of the memory unit do?’—and if they get a possible answer that saves the senior designer’s time, the tool is well worth it.”

He continued to tell EE Times, “I could imagine that [EDA tool vendors] would be very interested in having [an LLM] be a more approachable human interface to the tool. Writing scripts for a lot of these tools is an art, and if they could at least get a set of scripts for a particular tool and fine tune on that, then they could make it much easier for designers to use the tools because you could write in human language: ‘Here’s what I’d like the tool to do,’ and it would then generate the script that would direct the tool to do that.”

In the paper, Nvidia’s researchers evaluate their custom developed LLMs methods on three selected applications for chip design: an engineering assistant chatbot, EDA script generation, and bug summarization and analysis. They presented how their results show that the domain adaptation techniques they utilized enable significant LLM performance improvements over general-purpose base models across the three evaluated applications, enabling up to 5x model size reduction with similar or better performance on a range of design tasks. Their findings also indicated that there’s still room for improvement between current results and ideal outcomes. They conclude that further investigation of domain adapted LLM approaches will help close this gap in the future.

Lesson learned, the value of a customized LLM

The paper mainly focuses on the team’s work gathering its design data and using it to create a specialized generative AI model, a process portable to any industry. As its starting point, the team chose a foundation model and customized it with Nvidia NeMo, a framework for building, customizing and deploying generative AI models that’s included in the Nvidia AI Enterprise software platform. The selected NeMo model sports 43 billion parameters, a measure of its capability to understand patterns. It was trained using more than a trillion tokens, the words and symbols in text and software.

The team then refined the model in two training rounds, the first using about 24 billion tokens worth of its internal design data and the second on a mix of about 130,000 conversation and design examples.

The work is among several examples of research and proofs of concept of generative AI in the semiconductor industry, just beginning to emerge from the lab. One of the most important lessons this week’s research paper’s authors learned is the value of customizing an LLM.

On chip-design tasks, Nividia said custom ChipNeMo models with as few as 13 billion parameters match or exceed performance of even much larger general-purpose LLMs like LLaMA2 with 70 billion parameters. In some use cases, ChipNeMo models were dramatically better.

This article is written by Nitin Dahad and originally published on https://www.embedded.com/

Chip design assistant anyone?