Leaderboards
Benchmarks
Every score carries its origin.
Arena Hard Prompts168GPQA Diamond16LiveBench58LiveBench Data Analysis58LiveBench Reasoning58MMLU-Pro16
GPQA Diamond
% correct, graduate-level science questionsProvenance16 modelsscores Jun 12, 2024 – Jan 22, 2025
1Mistral LargeMistral AIScore43.7Score / $7ScoredNov 19, 20242Phi 4Microsoft14.7BScore40.6Score / $290Score / GB VRAM4ScoredJan 8, 20253Qwen2.5 72B InstructQwen72.7BScore37.5Score / $94Score / GB VRAM~0.82ScoredOct 16, 20244Gemma 2 27BGoogle27.2BScore37.5Score / $58Score / GB VRAM~2.2ScoredAug 7, 20245Mixtral 8x22B InstructMistral AI141BScore37.3Score / $6Score / GB VRAM~0.42ScoredJun 12, 20246Hermes 3 70B InstructNous Research70.6BScore36.2Score / $52Score / GB VRAM~0.81ScoredAug 28, 20247Llama 3.1 70B InstructMeta70.6BScore35.7Score / $89Score / GB VRAM~0.8ScoredAug 15, 20248Qwen2.5 Coder 32B InstructQwen32.8BScore34.9Score / $35Score / GB VRAM~1.7ScoredDec 10, 20249Llama 3.3 70B InstructMeta70.6BScore32.9Score / $103Score / GB VRAM~0.74ScoredDec 3, 202410Command R7B (12-2024)CohereScore30.9Score / $206ScoredDec 20, 202411Qwen2.5 7B InstructQwen7.6BScore29.1Score / $416Score / GB VRAM~6.1ScoredSep 18, 202412Mistral NemoMistral AI12.2BScore29Score / $967Score / GB VRAM~3.8ScoredAug 29, 202413Ministral 8BMistral AIScore28.4Score / $258ScoredDec 1, 202414Llama 3.2 3B InstructMeta3.2BScore27.9Score / $558Score / GB VRAM~14ScoredSep 27, 202415Llama 3.2 1B InstructMeta1.2BScore27.5Score / $1375Score / GB VRAM~36ScoredSep 23, 202416R1 Distill Llama 70BDeepSeek70.6BScore26.5Score / $33Score / GB VRAM~0.6ScoredJan 22, 2025
