feat: add inference.empty_cache_per_design flag to reduce CUDA allocator fragmentation#451

Merged

lyskov merged 2 commits intoRosettaCommons:mainfrom

leonardomarino:fix/empty-cache-per-design

Apr 24, 2026

Commits on Apr 21, 2026

feat: add inference.empty_cache_per_design flag (off by default)
leonardomarino
committed
feat: call torch.cuda.empty_cache() per design when empty_cache_per_design=True
leonardomarino
committed