CS 639 HW5 Writeup: Alignment

Content locked

The catalog is public. Reading this content requires approved access.

Part A: Core Alignment Implementation

A7: Training and Validation

Part B: Alignment Experiments

B1: Varying the Amount of Unsupported Preference Data

B2: Varying Preference Noise

Part C: DPO Comparison and Failure Analysis

C1: SFT vs. Reward-Model Reranking vs. DPO

C2: DPO Loss and Training