Performance drops in CMA and LDO

[zhsdcbj]

Dear author, I encountered some issues while running the training for the four circuits using your code. The TSA circuit training is working normally; it reached a success rate of around 90% after 12,000 steps, which is consistent with the paper. However, I am observing performance drops in the other circuits: LDO: It only achieved a success rate of around 60% after 12,000 steps. CMA: It only reached a success rate of around 10% after 24,000 steps, which is a significant gap compared to the results reported in the paper. Could you please advise on what might be causing these discrepancies? Thank you for your help.

[SeunggeunKimkr]

Hi, thanks for the detailed report.
Yes,performance can vary significantly across random seeds. RL training from scratch is typically much less stable than supervised learning, so single-seed results may not be representative. We recommend re-running the experiment with multiple seeds and comparing the aggregate statistics.
For reference, all metrics reported in the paper are averaged over 10 independent runs with different random seeds.

[zhsdcbj]

Thank you for your response. Following up, I conducted several experiments using different random seeds. However, I found that the overall success rate remains significantly lower than the results reported in the paper. Specifically, I ran 9 trials for the LDO circuit and 3 trials for the CMA circuit. The results are illustrated in the three figures below.

[zhsdcbj]

I replicated the test on a different machine, and the performance of both circuits is still unsatisfactory. I am using ngspice version 42.

[SeunggeunKimkr]

Thanks for sharing your results.
In my setup, I’m not seeing a success rate that low. Could you try adjusting the hyperparameters and re-running?
To help with reproduction, here are my W&B experiment results for the LDO runs:
https://api.wandb.ai/links/seunggeun_kim_kr-austin/a4xyhmos

[zhsdcbj]

Thank you for your response and for providing the data.
I would like to ask a question regarding your experiment naming conventions: specifically, which hyperparameters do terms like 'np12' and 'filter0' refer to?
Additionally, if it is convenient, would you be able to share the data for the CMA circuit? I would greatly appreciate it if you could.

[zjm200328-sudo]

Dear author, I am currently reproducing your project and have encountered some issues. Could you please provide your ngspice version, as well as share the W&B experiment logs or reference data for CMA, COMP, and TSA? I look forward to your reply.

[SeunggeunKimkr]

Dear all, we no longer have the W&B experiment logs.
The ngspice version we used was v31.

[SeunggeunKimkr]

Thank you for your response and for providing the data. I would like to ask a question regarding your experiment naming conventions: specifically, which hyperparameters do terms like 'np12' and 'filter0' refer to? Additionally, if it is convenient, would you be able to share the data for the CMA circuit? I would greatly appreciate it if you could.

Sorry for the late reply. we no longer have the W&B experiment logs. I guess 'np12_filter0' and 'is11t5_16_180' are just artifacts that we no longer use.

[zjm200328-sudo]

Dear author, thank you for your reply. I am currently using ngspice31 to reproduce your project, but when I switch to ngspice31 to run the LDO, I keep encountering issues: either "DC sweep output is shorter than expected" or "missing or empty simulation output", which causes the training to terminate. Have you encountered similar issues?