ChatGPT and Verilog RTL

A month ago, I heard about ChatGPT. I wanted to know how it will create Verilog RTL code. When I got to log in from their waitlist, I tried asking simple questions. I was impressed by the ability to spit out pretty clean and well-documented Verilog code. In this article, I'm documenting my brief experimentation with ChatGPT, results, and comments to provide a balanced picture. 

Here is a shameless plug. All these questions discussed here are taken from my interview questions blog page. Check it out at rajesh52.blogspot.com . I have almost 47 questions listed about ASIC/FPGA's front-end design. All these questions require some thinking and awareness about ASIC design flow and physical implementation understanding.  

I have conducted many interviews in the last 25 years of my career. I rarely find anybody giving a completely satisfactory answer to arbiter design questions. So I decided to pose that question first. I was surprised to get a solution that made sense on the surface. I did not check for the accuracy of the answer yet, but it made sense.

What impressed me was that the code was very well formatted and documented. It is rare to find a human engineer who does this consistently.

It included more port list descriptions, which was a pleasant surprise. Such details can be used to fill up design documentation. In the past, I have seen many engineers ignoring this. 

I decided to trust the code and venture further.

I moved to the latest question, #47, Multi clock sync pulse. I conveyed the intention of the interview question to ChatGPT.  

https://rajesh52.blogspot.com/2020/10/interview-question-47-multi-clock-sync.html 

RA: Design a Synchronizer module. There are two clocks for this module. The first clock is clockA and the second clock is clockB. 

The input inA is synchronous to clockA. The output outB is synchronous to clockB.

If inA gets asserted continuously for some number of clockA pulses then the output outB should also get asserted in clocks domain clockB for the same number of clockB clocks. The number of clocks for which inA gets asserted is variable and depends on external conditions. It can not be used as parameter as parameter is fixed for an instance. 

The goal is to design logic for output port outB.

Create Verilog code for module which behaves as instructed.

The port list seemed OK.

It created counters in two separate clock domains, which seemed fine.

The comparison of registers from two different clocks alarmed me. I started looking at the generated code with suspicion. 

There were many issues.

1. It was generating a fixed number of clocks (though parametric)

2. The counters were free running instead of depending on the input signal.

In short, the solution was incorrect and unusable. It was clear that ChatGPT did not understand or I could not convey my intention correctly. I tried 3 times more to correct my prompt, and the results got weirder and weirder. It even suggested a solution with shift registers.

I was at the end of my wits. I decided to give it an explicit prompt. 

------------------

Design a Synchronizer module. There are two clocks for this module. The first clock is clockA and the second clock is clockB. 

The input inA is synchronous to clockA. The output outB is synchronous to clockB.

Have a counter in clock domain clockA which counts the number of clocks for which inA was asserted. Synchonize this counter value to ClockB clock domain. In ClockB clock domain outB gets asserted as soon as it finds non-zero value of the count and decrements the count value. It gets deasserted when count reaches zero.

-----------------

OK. Some progress, but it ignored my synchronizing prompt. I am quickly losing faith in the "I" part of AI. 

This question is about synchronization, and later mentioning unrelated frequencies leads to the importance of Async FIFOs. ChatGPT behaved like a rookie Intern who is fast and confident but stupid. If the solution is available in the public domain, it can copy and present but lacks originality in problem-solving.

I decided to give up and try something else.

With lowered expectations, I posed another question of finding a faster clock. 

https://rajesh52.blogspot.com/2020/08/interview-question-45-find-faster-clock.html

It produced following RTL.

Not Bad, I thought, for a rookie Intern. I decided to give him a further follow-up question.

It figured out the module name and port list correctly. The intention of counting clocks for a fixed, known period seems legit.

 

I scratched my head and tried simulating in my head. The answer will be 1 for just one clock. Do you see something else happening?

Now I have concluded that ChatGPT is good enough to find well-known solutions and jumpstart writing some code, but accuracy is highly doubtful. It is awful in figuring out solutions to new problems. It might suggest a few answers to lead you off, but not good enough to provide them. 

In short, my brothers and sisters, your jobs are safe :)

I decided to test its strengths in mundane jobs where humans suck big time or get tired of uncreative tasks like reverse engineering. I opened a new window to ensure that previous discussions were not influencing the current job. I recreated the jumbled-up code from open-source random code.

I found this code here. https://www.fpga4student.com/2017/04/verilog-code-for-16-bit-risc-processor.html 

I removed all the comments, jumbled them up, and asked ChatGPT to indent and provide comments.

Here is the result,

It discovered the functionality.  The port list comments are better than the original.

Here are further comments

I am really impressed. This is a well-known function/component in open source. I agree. I set up my expectation lower. Maybe I secretly wanted ChatGPT to shine this time :)

I tried one more interesting task. Converting VHDL to Verilog.
I asked it to convert the Shifter VHDL code from this page.

https://www.fpga4student.com/2017/07/shifter-design-in-vhdl.html

I am really impressed. I won't trust it and will prefer to write a test bench to ensure the intention is correct, but the seemingly quality of the code is impressive.

Now I wanted to test its ability to generate test benches. First, I asked it to create a well-known Garage door opener design.

It spits out lengthy code.

Now I have warmed up to ChatGPT and almost started to believe it.

It provided a long nice looking test bench.

And then it did this.

Two initial statements running different tests and assigning values independently to the same variable. Nah, I lost confidence again. I can not trust ChatGPT to provide me trustworthy code. It acts like a very confident liar. You never know when you are getting duped.

I asked to rewrite the test bench in UVM. It gave a pretty elaborate test bench. I am not a UVM expert, so I won't comment on the correctness of testbench, but the sheer ability to spit out a test bench for a given module gives a jump start in writing a test bench from scratch.

I asked to provide an SDC file for previously generated clock_frequency_detector RTL. The result was simple but a good starting point if you need something to start.

ChatGPT is a worthy helper. It can speed up your productivity, it seems now.

As an RTL designer, I need some scripting help. I usually ask my colleagues with scripting experience or software folks. I decided to check out if ChatGPT can help me in that department.

Here is an example of a simple scripting task. I have a colossal netlist. I found a bug in a small unit that is instantiated many times. I fixed that small RTL block and wanted to run the same targeted test on the ECOed netlist. It is challenging to navigate a huge netlist to find out the exact location of that module in a hierarchical netlist. I want to break it up into smaller modules. 

Once I hack the netlist module and establish equivalence, I can combine it to get the larger netlist.

I asked the following to ChatGPT.

Wonderful! 

I asked it to run the script on the sample netlist I cludged together.

This is precisely what I wanted. I am impressed by the ability to write scripts. 

Final thoughts: ChatGPT is relatively new, and a new subject of Prompt Engineering is still taking shape. Here is a summary of my observations.

• ChatGPT is a clever, confident liar regarding pure RTL design.
• It lacks physical and timing awareness of hardware when solving problems.
• It is good at finding and presenting well-known designs
• It is incredible in reverse engineering, rewriting code with indentation, and generating comments.
• It can help in jumpstarting synthesis and STA scripts.
• It was terrific in assisting the hardware engineer by writing helpful scripts. 

I will continue to monitor the efficacy of ChatGPT for ASIC design. If you find compelling examples showing how it failed miserably or exceeded your expectations, please let me know by leaving comments.