Philip Tannor, Deepchecks, Co-Founder & CEO
Shir Chorev, Deepchecks, Co-founder & CTO

In this talk, we'll delve into the methods, metrics, and insights behind popular LLM Benchmarks, and learn how to review them effectively. We’ll take a close look at some of the notable leaderboards and LLM benchmarks (such as MMLU, HellSwag, TruthfulQA, MT-Bench) and understand their complexities and uniqueness. Ultimately, we’ll answer the question: did Gemini really outperform GPT4?
Finally, we’ll differentiate between evaluating LLM models to evaluating LLM based applications, and connect the discussion back to practical, real-world applications.