sp26-cs639 lecturetransformersattentionself-attentioncs639
Lecture 7: Transformers & Attention I Lecture 7: Transformers & Attention I
Content locked
The catalog is public. Reading this content requires approved access.
Sign in to readLecture 7: Transformers & Attention I
Basic Attention
History of Attention
Self-Attention: Motivation
Building Layers
Word Embeddings
Goals and Inputs
Retrieval Intuition
Query, Key, Value Vectors
Score Functions
Scoring and Scaling
Matrix Formulas
Additional Elements
Multi-Head Attention
Positional Encodings
Modern Positional Encodings
Transformers
Model Architecture
Architecture Overview
Inside an Encoder
Inside an Encoder: Details
Residual Connections and Layer Normalization
Inside a Decoder
Last Layers
Putting it All Together
The Rest