# Lecture 7 - Memory: TLBs, Smaller Page Tables

Source: https://wisconsin.twango.dev/sp26-cs537/lectures/lecture-07

# Lecture 7 - Memory: TLBs, Smaller Page Tables

**Instructor:** Michael Swift
**Topics:** TLBs, paging performance, workload locality, ASID, multilevel page tables, x86-64 paging

> [!abstract]- Learning Outcomes
> - Identify the challenges with paging
> - Explain how TLBs address the performance problem of paging
> - Explain how multilevel page tables address the space problem of paging

---

## Recap: Address Space with Paging

The virtual address is split into a **page number** and an **offset**. The page table maps each virtual page number (VPN) to a physical page frame number (PFN). Unused virtual pages have `null` entries in the page table.

- The **Page Table Base Register (PTBR)** points to the start of the page table in physical memory
- Each process has its own page table
- Virtual memory is contiguous per-process, but physical pages can be anywhere

### Growing Segments

When a segment (e.g., stack) grows to need more pages:

1. The OS allocates new physical pages wherever there is room
2. New page table entries are added mapping the new virtual pages to the new physical frames
3. The new physical pages do **not** need to be contiguous — paging eliminates external fragmentation

> [!note] Lecture Note
> If the stack grows to need more space, the OS simply allocates new pages wherever there is room in physical memory and updates the page table. No contiguous allocation needed.

---

## Translation Steps with Paging

For each memory reference:

1. Extract **VPN** (virtual page number) from the **VA** (virtual address)
2. Calculate address of **PTE** (page table entry): `PTBR + VPN * PTE_size`
3. Read PTE from memory; check present bit
4. Extract **PFN** (page frame number) from PTE
5. Build **PA** (physical address): `PFN | offset`
6. Read contents of PA from memory

> [!example] Example
> Given: 14-bit addresses, PTBR = `0x5000`, PTE size = 4 bytes, 4KB pages (12-bit offset)
>
> Page table: `[2, 7, 80, 99]` (index = VPN, value = PFN)
>
> `READ 0x1103`:
> - VPN = `0x1` (upper 2 bits), offset = `0x103`
> - PTE address = `0x5000 + 1 * 4 = 0x5004`
> - PFN = 7
> - PA = `0x7103`

---

## Problem: Paging Doubles Memory Accesses

Every memory access now requires **two** physical memory accesses: one to read the page table entry, and one to read/write the actual data.

> [!example] Example
> ```text
> 0x0010: movl  0x1100, %edi
> 0x0013: addl  $0x3, %edi
> 0x0019: movl  %edi, 0x1100
> ```
>
> Assume PTBR = `0x5000`, PTE size = 4 bytes, 4KB pages.
> Page table: `[2, 0, 80, 99]`
>
> With **segmentation**: 5 memory references (3 instruction fetches + 2 movl data accesses)
>
> With **paging**:
> 1. Fetch instruction at `0x0010` (VPN 0) — mem ref 1: read PTE at `0x5000`, mem ref 2: fetch from `0x2010`
> 2. Execute `movl` load from `0x1100` (VPN 1) — mem ref 3: read PTE at `0x5004`, mem ref 4: load from `0x0100`
> 3. ...and so on for each instruction and data access
>
> **Page table is slow — it doubles memory references!**

---

## TLB (Translation Lookaside Buffer)

> **TLB (Translation Lookaside Buffer):** A hardware cache on the CPU that stores recent virtual-to-physical page translations, avoiding the need to access the page table in memory on every memory reference.

### Strategy: Cache Page Translations

- The page table lives in **RAM**
- The CPU contains a small **translation cache** (TLB)
- The **MMU** performs a TLB lookup on every memory access
- On a **TLB hit**, the physical page number is obtained without accessing memory
- On a **TLB miss**, the page table in memory is consulted and the TLB is updated

### TLB Sequential Access Example

With 4KB pages and a process P1 with page table `[1, 5, 4, ...]`:

| Virtual Access | Action | Physical Access |
|---|---|---|
| `load 0x1000` | TLB miss, load PTE from `0x0004`, learn VPN 1 → PPN 5 | `load 0x5000` |
| `load 0x1004` | TLB hit | `load 0x5004` |
| `load 0x1008` | TLB hit | `load 0x5008` |
| `load 0x100C` | TLB hit | `load 0x500C` |
| ... | | |
| `load 0x2000` | TLB miss, load PTE from `0x0008`, learn VPN 2 → PPN 4 | `load 0x4000` |
| `load 0x2004` | TLB hit | `load 0x4004` |

> [!note] Lecture Note
> The first access to a new page causes a TLB miss (2 memory accesses). Subsequent accesses to the same page are TLB hits (1 memory access). Sequential access patterns achieve very high TLB hit rates.

---

## TLB Performance

**Miss rate** = # TLB misses / # TLB lookups

> [!example] Example
> ```c
> int sum = 0;
> for (i=0; i<2048; i++) {
>     sum += a[i];
> }
> ```
>
> - `int` is 4 bytes, page size is 4KB
> - TLB lookups = 2048 (one per array access)
> - TLB misses = number of unique pages accessed = 2048 * 4 / 4096 = 2 pages → **2 misses**
> - Miss rate = 2/2048 ≈ **0.1%**
> - Hit rate = 2046/2048 ≈ **99.9%**

> [!note] Lecture Note
> With smaller pages, the hit rate would get **worse** because each page covers fewer array elements, leading to more misses.

### Workload Access Patterns

| Workload A (Sequential) | Workload B (Random) |
|---|---|
| `for (i=0; i<2048; i++) sum += a[i];` | `sum += a[rand() % N];` |
| Good for TLB — accesses are to nearby addresses on the same page | Bad for TLB (depends on size of N) — random accesses may hit many different pages |

### Workload Locality

> **Spatial Locality:** Future accesses will be to nearby addresses.

> **Temporal Locality:** Future accesses will be repeats to the same data.

| | Spatial Locality | Temporal Locality |
|---|---|---|
| TLB behavior | Access same page repeatedly; same VPN→PPN translation reused | Access same address near in future; same TLB entry reused |
| What helps | **Larger pages** | **More TLB entries** |

### Improving TLB Performance

**Increase number of entries:**
- Add second-level TLB — 1536 entries on Intel (L1 TLB is small and fast, L2 TLB is larger and slower)

**Increase page size:**
- Fewer unique translations needed to access the same amount of memory
- Example: accessing a 1 MB array
  - 4KB pages: 256 TLB misses
  - 256KB pages: 4 TLB misses
- Intel/AMD x86-64 supports: **4KB**, **2MB**, and **1GB** pages

---

## TLB and Context Switches

What happens if a process uses cached TLB entries from another process?

**Option 1: Flush TLB on each context switch**
- Costly — lose all recently cached translations

**Option 2: Track which entries belong to which process**
- **Address Space Identifier (ASID)** — 8 to 16 bits (256 – 65,536 entries)
- Tag each TLB entry with an ASID
- Can set ASID = PID for few processes

> [!example] Example with ASID
> P1 page table (ASID 11): `[1, 5, 4, ...]`
> P2 page table (ASID 12): `[6, 2, 3, ...]`
>
> | Valid | Virt | Phys | ASID |
> |---|---|---|---|
> | 0 | 1 | 9 | 11 |
> | 1 | 1 | 5 | 11 |
> | 1 | 1 | 2 | 12 |
> | 1 | 0 | 1 | 11 |
>
> `load 0x1444` with ASID 12 → physical page **2**, PA = `0x2444`
> `load 0x1444` with ASID 11 → physical page **5**, PA = `0x5444`
>
> Same virtual address maps to different physical addresses based on ASID.

---

## HW and OS Roles for Paging

**CPU must know where page tables are:**
- Page Table Base Register is called **`CR3`** register on x86
- Page table structure is fixed and agreed upon between HW and OS

**OS builds page table, fills in entries:**
- Page table is stored with physical addresses

**HW "walks" the page table on TLB miss and fills TLB:**
- Evicts LRU entry to make space

### OS Updating the Page Table

OS must **invalidate TLB entries** when PTEs change (using `invlpg` instruction on x86):

| Program | OS | TLB |
|---|---|---|
| `Load 0x3250000` | `PTE[0x3250].PPN = 0x2456` | `PTE[0x3250] = 0x2456` |
| `Free(0x3250000)` | `PTE[0x3250].val = 0` then **`invlpg 0x3250`** | `PTE[0x3250] = nil` |
| `Load 0x3250000` | `PTE[0x6434].PPN = 0x2456` | (miss, re-fetch from page table) |

---

## TLB Summary

- Pages are great, but accessing page tables for every memory access is slow
- Cache recent page translations → **TLB**
  - MMU performs TLB lookup on every memory access
- TLB performance depends strongly on workload
  - Sequential workloads perform well
  - Workloads with temporal locality can perform well
  - Larger pages help spatial locality
- TLBs increase cost of context switches
  - Flush TLB on every context switch, or
  - Add ASID to every TLB entry

---

## Smaller Page Tables

### The Problem: Page Table Size

The page table has one entry for every virtual page — most of which may be unused (null entries for gaps between code, data, heap, and stack).

**32-bit address space with 4KB pages:**
- Page size = 4KB = 2^12 → 12-bit offset
- Number of PTEs = 2^32 / 2^12 = **2^20** (about a million entries)
- At 4 bytes each → **4 MiB** per process

**64-bit address space (x86_64) with 4KB pages:**
- Number of PTEs = 2^64 / 2^12 = **2^52** (4.5 x 10^15 entries)
- At 8 bytes each → **36 exabytes** per process — a ridiculous amount!

Mostly, the address space is *sparse* (has holes that are not mapped to physical memory), so most of this space is taken up by page tables mapped to nothing.

---

## Multilevel Page Tables

**Goal:** Allow page table to be allocated non-contiguously.

**Idea:** Page the page tables themselves.

- Creates multiple levels of page tables; outer level is the **page directory**
- Only allocate inner page tables for virtual address ranges actually in use
- Used in x86 architectures (hardware can walk the known structure)

### Radix Tree Structure

For a 20-bit address with 4KB pages (12-bit offset):

| outer page (4 bits) | inner page (4 bits) | page offset (12 bits) |
|---|---|---|

- The **page directory** (outer level) has 2^4 = 16 entries
- Each entry points to an **inner page table** (or is null if that region is unused)
- Each inner page table has 2^4 = 16 entries mapping to physical page frames

> [!note] Lecture Note
> Invalid entries in the page directory (marked with X) mean that entire inner page table does not exist, saving memory.

### Sizing the Address Bits

**Goal:** Each inner page table should fit within a single page.

**Constraint:** PTE size * number of PTEs per table = page size

> [!example] Example: 30-bit address, 4KB pages, 4-byte PTEs
> - Page size = 2^12 bytes = 4KB
> - PTEs per page = 2^12 / 2^2 = 2^10 = 1024 entries
> - Bits for inner page index = log2(1024) = **10 bits**
> - Remaining bits for outer page = 30 - 10 - 12 = **8 bits**

---

## Multilevel Translation Example

20-bit address: outer page (4 bits) | inner page (4 bits) | page offset (12 bits)

| Page Directory | | Page of PT (@PPN:0x3) | | Page of PT (@PPN:0x92) | |
|---|---|---|---|---|---|
| **PPN** | **valid** | **PPN** | **valid** | **PPN** | **valid** |
| 0x3 | 1 | 0x10 | 1 | - | 0 |
| - | 0 | 0x23 | 1 | - | 0 |
| - | 0 | - | 0 | - | 0 |
| - | 0 | - | 0 | - | 0 |
| - | 0 | 0x80 | 1 | - | 0 |
| - | 0 | 0x59 | 1 | - | 0 |
| ... | ... | ... | ... | ... | ... |
| 0x92 | 1 | - | 0 | 0x55 | 1 |
| | | | | 0x45 | 1 |

> [!example] Translate `0x01ABC`
> - Outer page = `0x0` → page directory entry 0 → PPN = `0x3` (valid)
> - Inner page = `0x1` → page table at PPN `0x3`, entry 1 → PPN = `0x23` (valid)
> - Offset = `0xABC`
> - **Physical address = `0x23ABC`**

> [!example] Translate `0xFEED0`
> - Outer page = `0xF` → page directory entry 15 → PPN = `0x92` (valid)
> - Inner page = `0xE` → page table at PPN `0x92`, entry 14 → PPN = `0x55` (valid)
> - Offset = `0xED0`
> - **Physical address = `0x55ED0`**
> - Memory accesses needed for translation: **2** (one for page directory, one for inner page table)

---

## Practice: Multilevel Page Tables

> [!example] Practice #1
> Consider a virtual address space of **16KB** with **64-byte** pages.
>
> 1. How many bits in the virtual address? 16KB = 2^14 → **14 bits**
> 2. Total entries in a linear page table? Page size = 2^6, so 2^14 / 2^6 = 2^8 = **256 entries**
> 3. Two-level page table with 4-byte PTEs — how many bits for the inner page index?
>    PTEs per page = 64 / 4 = 16 = 2^4 → **4 bits** for inner page
>    Remaining for outer page = 8 - 4 = **4 bits**

---

## Problem with 2 Levels

With a **64-bit** address space, a two-level page table still has an outer page directory that is too large to fit in a single page.

**Solution:** Split the page directory into pieces — add more levels.

| PD idx 0 | PD idx 1 | PT idx | page offset |
|---|---|---|---|

With 4KB pages and 4-byte PTEs → 4KB / 4 bytes = **1K entries per level**

| Levels | Virtual Address Space |
|---|---|
| 1 level | 1K * 4KB = **4 MB** |
| 2 levels | 1K * 1K * 4KB = **4 GB** |
| 3 levels | 1K * 1K * 1K * 4KB = **4 TB** |

---

## x86-64 Paging

- **Virtual address:** 48 bits (upper 16 bits are sign extended — all 0 or all 1)
- **Physical address:** 48 bits
- **PTE size:** 48 bits + metadata = **8 bytes**
- **PTEs per 4KB page:** 4096 / 8 = **512** = 2^9 → **9 bits** per level

**4-level page table structure (48-bit virtual address):**

| Bits 47–39 | Bits 38–30 | Bits 29–21 | Bits 21–12 | Bits 11–0 |
|---|---|---|---|---|
| Page Map Lvl 4 (9 bits) | Page Pointer Dir. (9 bits) | Page Directory (9 bits) | Page Table (9 bits) | Offset (12 bits) |

- 4 * 9 = 36 bits for page table indices + 12 bits offset = **48 bits**

### x86-64 Translation

Translation walks **4 levels** of page tables starting from the `CR3` register:

1. `CR3` → Page Map Level 4 entry (mem access 1)
2. → Page Pointer Directory entry (mem access 2)
3. → Page Directory entry (mem access 3)
4. → Page Table entry (mem access 4)
5. → Physical address

> [!note] Lecture Note
> Each translation requires **4 memory accesses** to walk the page table (without TLB). This makes the TLB even more critical for performance on x86-64.

---

## Related

- **Navigation:** [[lecture-06|← Lecture 6]] · [[lecture-08|Lecture 8 →]]
- **Exam:** [[exams/midterm-1/index|Midterm 1]]
- **Course:** [[README|CS 537 Map of Content]]