Add scaling_bench: encode_batch vs worker-pool comparison (#1900)#2048

stargazerZJ wants to merge 1 commit into

huggingface:mainhuggingface/tokenizers:mainfrom

stargazerZJ:scaling-benchstargazerZJ/tokenizers:scaling-benchCopy head branch name to clipboard