sp26-cs639 lecturebertgptllamacs639
Lecture 10: Architectures II Lecture 10: Architectures II
Content locked
The catalog is public. Reading this content requires approved access.
Sign in to readLecture 10: Architectures II
Encoder-Only Models: BERT
Why Encoder-Decoder?
Architecture
Forward Pass
Training
Task 1: Masked Language Modeling
Task 2: Next Sentence Prediction
Training Details
Fine-Tuning
Decoder-Only Models: GPT
Architecture
GPT Acronym
Causal Masking
Autoregressive Next Token Prediction
From GPT-2 to GPT-3
Open Source: Llama 3.1
Improvements