Optimise dataframe_to_mds for Large-Scale Datasets via intermediate Aggressive Disk Cleanup and Distributed Metadata Handling - #971

Optimise dataframe_to_mds for Large-Scale Datasets via intermediate Aggressive Disk Cleanup and Distributed Metadata Handling#971

AbhishekKumarSingh wants to merge 1 commit into

mosaicml:mainmosaicml/streaming:mainfrom

AbhishekKumarSingh:converter_fixAbhishekKumarSingh/streaming:converter_fixCopy head branch name to clipboard