README

Content locked

The catalog is public. Reading this content requires approved access.

CS 639 Assignment 2: Transformers

Google Colab Setup

Estimated Run Time

Submission

Model Configuration

Part A: Core Transformer Implementation (60%)

A1: Scaled Dot-Product Attention (15 points) [Code]

A2: Multi-Head Attention (15 points) [Code]

A3: Transformer Block (15 points) [Code]

A4: GPT Forward Pass (10 points) [Code]

A5: Training and Validation (5 points) [Both]

Part B: Ablation Experiments (10%)

B1: Varying Number of Attention Heads (5 points) [Writeup]

B2: Varying Number of Layers (5 points) [Writeup]

Part C: Tokenization & Positional Encoding (30%)

C1: Character-Level vs. Word-Level Tokenization (5 points) [Writeup]

C2: Rotary Positional Embeddings (RoPE) (25 points) [Both]