The Evolving Landscape of LLM Evaluation
Throughout recent years, LLM capabilities have outpaced evaluation benchmarks. This is not a new development. The set of canonical LLM evals has further narrowed to a small set of benchmarks such...
// Latest
From Sebastian Ruder
5 posts
Command R+
Update August 30, 2024: We've released updated versions of Command R and R+.This post is an update on what I’ve been up to since I joined Cohere. I’ve had fun contributing to the launch...
True Zero-shot MT
Little over a week ago, Gemini 1.5 reported close to human-level performance on MTOB, a recent challenging translation dataset. In this post, we’ll dig into this result, explore true zero-shot...
Thoughts on the 2024 AI Job Market
It’s crazy how the AI and NLP landscape has evolved over the last five years. 5 years ago, around the time I finished my PhD, if you wanted to work on cutting-edge natural language processing (NLP), your choice...
The Big Picture of AI Research
More papers on AI are published than ever before but each paper tends to only present its part of the picture—and it becomes difficult to recognize the larger story to which a paper is connected. To...