Clayton's Blog

How I Sped Up the ChatBot Arena Ratings Calculations from 19 Minutes to 8 Seconds

TLDR: 🔗 Vectorized preprocessing Deduplication and sample weighting Multinomial bootstrapping Avoid recomputation Exploit sparsity Exploit symmetry Multiprocessing Optimize in-memory data layout Read the code: PR: Accelerate Bradley Terry MLE model fitting What is ChatBot Arena? 🔗ChatBot Arena is a website developed by LMSYS where users can put in their own prompts and get replies from two random anonymized LLMs. The user can then vote for which response they prefer or mark it as a tie.

test blog

Tags: test
please ignore