sp26-cs639 lecturetransformersattentioncross-attentiondecoder-maskingcs639
Lecture 8: Transformers & Attention II Lecture 8: Transformers & Attention II
Content locked
The catalog is public. Reading this content requires approved access.
Sign in to readLecture 8: Transformers & Attention II
Review: Basic Attention
Building Layers
Goals and Inputs
Retrieval Intuition
Query, Key, Value Vectors
Score Functions
Scoring and Scaling
Putting it Together
Matrix Formulas
Additional Elements
Multi-Head Attention
Positional Encodings
Modern Positional Encodings
Full Architecture
Model Architecture
Architecture Overview
Inside an Encoder
Residual Connections and Layer Norm
Inside a Decoder
Cross-Attention
Decoder Masking
Last Layers
Putting it All Together
Training
The Rest