sp26-cs639 lecturetransformersbertgptcs639
Lecture 9: Architectures I Lecture 9: Architectures I
Content locked
The catalog is public. Reading this content requires approved access.
Sign in to readLecture 9: Architectures I
Review: Full Transformer Architecture
Model Architecture
Architecture
Inside an Encoder
Residual Connections and Layer Normalization
Inside a Decoder
Cross-Attention
Decoder Masking
Last Layers
Putting it All Together
Training
Encoder-Only Models: BERT
Why Encoder-Decoder?
Architecture
Forward Pass
Training
Task 1: Masked Language Modeling
Task 2: Next Sentence Prediction
Decoder-Only Models: GPT
Architecture