Lecture 7 - Memory: TLBs, Smaller Page Tables

Instructor: Michael Swift Topics: TLBs, paging performance, workload locality, ASID, multilevel page tables, x86-64 paging

Learning Outcomes

• Identify the challenges with paging
• Explain how TLBs address the performance problem of paging
• Explain how multilevel page tables address the space problem of paging

Recap: Address Space with Paging

The virtual address is split into a page number and an offset. The page table maps each virtual page number (VPN) to a physical page frame number (PFN). Unused virtual pages have null entries in the page table.

• The Page Table Base Register (PTBR) points to the start of the page table in physical memory
• Each process has its own page table
• Virtual memory is contiguous per-process, but physical pages can be anywhere

Growing Segments

When a segment (e.g., stack) grows to need more pages:

1. The OS allocates new physical pages wherever there is room
2. New page table entries are added mapping the new virtual pages to the new physical frames
3. The new physical pages do not need to be contiguous — paging eliminates external fragmentation

Lecture Note

If the stack grows to need more space, the OS simply allocates new pages wherever there is room in physical memory and updates the page table. No contiguous allocation needed.

Translation Steps with Paging

For each memory reference:

1. Extract VPN (virtual page number) from the VA (virtual address)
2. Calculate address of PTE (page table entry): PTBR + VPN * PTE_size
3. Read PTE from memory; check present bit
4. Extract PFN (page frame number) from PTE
5. Build PA (physical address): PFN | offset
6. Read contents of PA from memory

Example

Given: 14-bit addresses, PTBR = 0x5000, PTE size = 4 bytes, 4KB pages (12-bit offset)

Page table: [2, 7, 80, 99] (index = VPN, value = PFN)

READ 0x1103:

• VPN = 0x1 (upper 2 bits), offset = 0x103
• PTE address = 0x5000 + 1 * 4 = 0x5004
• PFN = 7
• PA = 0x7103

Problem: Paging Doubles Memory Accesses

Every memory access now requires two physical memory accesses: one to read the page table entry, and one to read/write the actual data.

Example

0x0010: movl  0x1100, %edi
0x0013: addl  $0x3, %edi
0x0019: movl  %edi, 0x1100

Assume PTBR = 0x5000, PTE size = 4 bytes, 4KB pages. Page table: [2, 0, 80, 99]

With segmentation: 5 memory references (3 instruction fetches + 2 movl data accesses)

With paging:

1. Fetch instruction at 0x0010 (VPN 0) — mem ref 1: read PTE at 0x5000, mem ref 2: fetch from 0x2010
2. Execute movl load from 0x1100 (VPN 1) — mem ref 3: read PTE at 0x5004, mem ref 4: load from 0x0100
3. …and so on for each instruction and data access

Page table is slow — it doubles memory references!

TLB (Translation Lookaside Buffer)

TLB (Translation Lookaside Buffer): A hardware cache on the CPU that stores recent virtual-to-physical page translations, avoiding the need to access the page table in memory on every memory reference.

Strategy: Cache Page Translations

• The page table lives in RAM
• The CPU contains a small translation cache (TLB)
• The MMU performs a TLB lookup on every memory access
• On a TLB hit, the physical page number is obtained without accessing memory
• On a TLB miss, the page table in memory is consulted and the TLB is updated

TLB Sequential Access Example

With 4KB pages and a process P1 with page table [1, 5, 4, ...]:

Lecture Note

The first access to a new page causes a TLB miss (2 memory accesses). Subsequent accesses to the same page are TLB hits (1 memory access). Sequential access patterns achieve very high TLB hit rates.

TLB Performance

Miss rate = # TLB misses / # TLB lookups

Example

int sum = 0;
for (i=0; i<2048; i++) {
    sum += a[i];
}

• int is 4 bytes, page size is 4KB
• TLB lookups = 2048 (one per array access)
• TLB misses = number of unique pages accessed = 2048 * 4 / 4096 = 2 pages → 2 misses
• Miss rate = 2/2048 ≈ 0.1%
• Hit rate = 2046/2048 ≈ 99.9%

Lecture Note

With smaller pages, the hit rate would get worse because each page covers fewer array elements, leading to more misses.

Workload Access Patterns

Workload Locality

Spatial Locality: Future accesses will be to nearby addresses.

Temporal Locality: Future accesses will be repeats to the same data.

Improving TLB Performance

Increase number of entries:

• Add second-level TLB — 1536 entries on Intel (L1 TLB is small and fast, L2 TLB is larger and slower)

Increase page size:

• Fewer unique translations needed to access the same amount of memory
• Example: accessing a 1 MB array
  • 4KB pages: 256 TLB misses
  • 256KB pages: 4 TLB misses
• Intel/AMD x86-64 supports: 4KB, 2MB, and 1GB pages

TLB and Context Switches

What happens if a process uses cached TLB entries from another process?

Option 1: Flush TLB on each context switch

• Costly — lose all recently cached translations

Option 2: Track which entries belong to which process

• Address Space Identifier (ASID) — 8 to 16 bits (256 – 65,536 entries)
• Tag each TLB entry with an ASID
• Can set ASID = PID for few processes

Example with ASID

P1 page table (ASID 11): [1, 5, 4, ...] P2 page table (ASID 12): [6, 2, 3, ...]

Valid	Virt	Phys	ASID
0	1	9	11
1	1	5	11
1	1	2	12
1	0	1	11

load 0x1444 with ASID 12 → physical page 2, PA = 0x2444 load 0x1444 with ASID 11 → physical page 5, PA = 0x5444

Same virtual address maps to different physical addresses based on ASID.

HW and OS Roles for Paging

CPU must know where page tables are:

• Page Table Base Register is called CR3 register on x86
• Page table structure is fixed and agreed upon between HW and OS

OS builds page table, fills in entries:

• Page table is stored with physical addresses

HW “walks” the page table on TLB miss and fills TLB:

• Evicts LRU entry to make space

OS Updating the Page Table

OS must invalidate TLB entries when PTEs change (using invlpg instruction on x86):

TLB Summary

• Pages are great, but accessing page tables for every memory access is slow
• Cache recent page translations → TLB
  • MMU performs TLB lookup on every memory access
• TLB performance depends strongly on workload
  • Sequential workloads perform well
  • Workloads with temporal locality can perform well
  • Larger pages help spatial locality
• TLBs increase cost of context switches
  • Flush TLB on every context switch, or
  • Add ASID to every TLB entry

Smaller Page Tables

The Problem: Page Table Size

The page table has one entry for every virtual page — most of which may be unused (null entries for gaps between code, data, heap, and stack).

32-bit address space with 4KB pages:

• Page size = 4KB = 2^12 → 12-bit offset
• Number of PTEs = 2^32 / 2^12 = 2^20 (about a million entries)
• At 4 bytes each → 4 MiB per process

64-bit address space (x86_64) with 4KB pages:

• Number of PTEs = 2^64 / 2^12 = 2^52 (4.5 x 10^15 entries)
• At 8 bytes each → 36 exabytes per process — a ridiculous amount!

Mostly, the address space is sparse (has holes that are not mapped to physical memory), so most of this space is taken up by page tables mapped to nothing.

Multilevel Page Tables

Goal: Allow page table to be allocated non-contiguously.

Idea: Page the page tables themselves.

• Creates multiple levels of page tables; outer level is the page directory
• Only allocate inner page tables for virtual address ranges actually in use
• Used in x86 architectures (hardware can walk the known structure)

Radix Tree Structure

For a 20-bit address with 4KB pages (12-bit offset):

• The page directory (outer level) has 2^4 = 16 entries
• Each entry points to an inner page table (or is null if that region is unused)
• Each inner page table has 2^4 = 16 entries mapping to physical page frames

Lecture Note

Invalid entries in the page directory (marked with X) mean that entire inner page table does not exist, saving memory.

Sizing the Address Bits

Goal: Each inner page table should fit within a single page.

Constraint: PTE size * number of PTEs per table = page size

Example: 30-bit address, 4KB pages, 4-byte PTEs

• Page size = 2^12 bytes = 4KB
• PTEs per page = 2^12 / 2^2 = 2^10 = 1024 entries
• Bits for inner page index = log2(1024) = 10 bits
• Remaining bits for outer page = 30 - 10 - 12 = 8 bits

Multilevel Translation Example

20-bit address: outer page (4 bits) | inner page (4 bits) | page offset (12 bits)

Translate 0x01ABC

• Outer page = 0x0 → page directory entry 0 → PPN = 0x3 (valid)
• Inner page = 0x1 → page table at PPN 0x3, entry 1 → PPN = 0x23 (valid)
• Offset = 0xABC
• Physical address = 0x23ABC

Translate 0xFEED0

• Outer page = 0xF → page directory entry 15 → PPN = 0x92 (valid)
• Inner page = 0xE → page table at PPN 0x92, entry 14 → PPN = 0x55 (valid)
• Offset = 0xED0
• Physical address = 0x55ED0
• Memory accesses needed for translation: 2 (one for page directory, one for inner page table)

Practice: Multilevel Page Tables

Practice #1

Consider a virtual address space of 16KB with 64-byte pages.

1. How many bits in the virtual address? 16KB = 2^14 → 14 bits
2. Total entries in a linear page table? Page size = 2^6, so 2^14 / 2^6 = 2^8 = 256 entries
3. Two-level page table with 4-byte PTEs — how many bits for the inner page index? PTEs per page = 64 / 4 = 16 = 2^4 → 4 bits for inner page Remaining for outer page = 8 - 4 = 4 bits

Problem with 2 Levels

With a 64-bit address space, a two-level page table still has an outer page directory that is too large to fit in a single page.

Solution: Split the page directory into pieces — add more levels.

With 4KB pages and 4-byte PTEs → 4KB / 4 bytes = 1K entries per level

x86-64 Paging

• Virtual address: 48 bits (upper 16 bits are sign extended — all 0 or all 1)
• Physical address: 48 bits
• PTE size: 48 bits + metadata = 8 bytes
• PTEs per 4KB page: 4096 / 8 = 512 = 2^9 → 9 bits per level

4-level page table structure (48-bit virtual address):

• 4 * 9 = 36 bits for page table indices + 12 bits offset = 48 bits

x86-64 Translation

Translation walks 4 levels of page tables starting from the CR3 register:

1. CR3 → Page Map Level 4 entry (mem access 1)
2. → Page Pointer Directory entry (mem access 2)
3. → Page Directory entry (mem access 3)
4. → Page Table entry (mem access 4)
5. → Physical address

Lecture Note

Each translation requires 4 memory accesses to walk the page table (without TLB). This makes the TLB even more critical for performance on x86-64.

Related

• Navigation: ← Lecture 6 · Lecture 8 →
• Exam: Midterm 1
• Course: CS 537 Map of Content