sp26-cs639 cs639
Lecture 21: Evaluation I Lecture 21: Evaluation I
Content locked
The catalog is public. Reading this content requires approved access.
Sign in to readLecture 21: Evaluation I
Evaluation needs a dataset, metric, and mechanism
HumanEval: evaluate generated programs by behavior
The pass@k metric
MMLU and its revisions
MMLU-Pro
MMLU-Redux
Holistic evaluation
LLM as judge
Biases and capability limits
Benchmarks can combine fixed tasks with model judging
Connections
Source figure