Lecture 18: Alignment II

Content locked

The catalog is public. Reading this content requires approved access.

What a reward model learns

Limitations occur at every stage

Human feedback is neither free nor automatically representative

A scalar reward is an imperfect specification

The learned policy inherits these choices

Direct preference optimization

Connections