fa25-cs540 cs540
Lecture 18: Deep Learning IV Lecture 18: Deep Learning IV
Content locked
The catalog is public. Reading this content requires approved access.
Sign in to readLecture 18: Deep Learning IV
The Attention Mechanism
How Attention Works
Matrix Notation for Attention
The Transformer Architecture
From Attention to Transformer
Multi-Head Attention
Positional Encoding
Encoder-Decoder Structure
Applications of Transformers
Data Concerns
The Overfitting Problem
Data Augmentation
Transformations for Images
Automating Augmentation
Augmentation for Text Data
Importance of Augmentation
Other Forms of Regularization
Classic Regularizations
Graph Neural Networks
Relationships in Data
Signal from Relationships
Graph Neural Network Architecture
How GCNs Work
Neural Networks Review
Perceptron
Multi-Layer Perceptron
Deep Neural Networks
Multi-Class Classification with Softmax
Training Neural Networks
Gradient Descent
Minibatch Stochastic Gradient Descent
Backpropagation
Convolutional Neural Networks
Why Not Fully Connected?
2-D Convolution
Stride and Padding
Multiple Channels
Evolution of CNN Architectures
Other Deep Architectures
Practice Problems
Key Takeaways
Suggested Reading