How I Sped Up the ChatBot Arena Ratings Calculations from 19 Minutes to 8 Seconds
TLDR: 🔗 Vectorized preprocessing Deduplication and sample weighting Multinomial bootstrapping Avoid recomputation Exploit sparsity Exploit symmetry Multiprocessing Optimize in-memory data layout Read the code: PR: Accelerate Bradley Terry MLE model fitting
What is ChatBot Arena? 🔗ChatBot Arena is a website developed by LMSYS where users can put in their own prompts and get replies from two random anonymized LLMs. The user can then vote for which response they prefer or mark it as a tie.
test blog
please ignore