sp26-cs639 lecturefoundation-modelspretrainingembeddingscs639
Lecture 1: Course Overview Lecture 1: Course Overview
Content locked
The catalog is public. Reading this content requires approved access.
Sign in to readLecture 1: Course Overview
Logistics
Lecture Location
Enrollment
Teaching Team
Content
Lecture Formats
Assignments & Grades
Class Setup
Reading
Background
Goals
What Is a Foundation Model?
Three Historical Trends
1. Multitask Models
What’s a Task?
Handling Multiple Tasks
An Alternative
2. Pretraining and Fine-tuning
3. Word Embeddings and Language Models
Word Embeddings
Contextual Embeddings
Translation and Transformers
Combining the Trends: BERT
Language Models
Summary