fa25-cs540 cs540
Lecture 17: Deep Learning III Lecture 17: Deep Learning III
Content locked
The catalog is public. Reading this content requires approved access.
Sign in to readLecture 17: Deep Learning III
Review: ResNets and Deep Networks
Evolution of CNN Architectures
The Problem with Adding More Layers
Residual Connections
Full ResNet Architecture
Additional ResNet Techniques
Recurrent Neural Networks (RNNs)
Why Recurrent Neural Networks?
RNN Architecture
RNNs: High Level
RNN Equations
RNNs for Language Modeling
Language Modeling
Next-Token Prediction
Example: RNNs on Text
The Attention Mechanism
From RNNs to Transformers
Word Representations and Context
Attempt 1: Naive Contextual Embedding
Attempt 2: Assigning Weights
The Attention Mechanism
Query, Key, and Value Matrices
Notation for Attention
The Transformer Architecture
From Attention to Transformer
Multi-Head Attention
Positional Encoding
Encoder-Decoder Structure
Decoder Details
Applications of Transformers
Practice Problems
Key Takeaways
Suggested Reading