Lecture 8 - Memory: Beyond Physical

Instructor: Michael Swift Topics: Large pages, paging tricks, shared memory, copy-on-write, swapping, page replacement policies, clock algorithm, thrashing, memory-mapped files

Learning Outcomes

• Understand how to use page tables to share data efficiently
• Understand how the kernel accesses user data
• Understand how swapping works
• Apply swapping policies to determine the number of page faults

Review: Multilevel Page Tables

Multilevel page tables use a radix tree data structure.

Example: 20-bit Address

With a 20-bit address space:

• Outer page (4 bits) — indexes into the page directory
• Inner page (4 bits) — indexes into a page table
• Page offset (12 bits) — offset within the page

The base of the page directory points to the outer table; each valid entry points to an inner page table, which in turn points to the physical page.

Practice: Translating 0xBEED0

Given a 2-level page table with 4-bit outer and 4-bit inner indices (12-bit offset):

• 0xBEED0 → outer index = 0xB, inner index = 0xE, offset = 0xED0
• Page directory entry at index 0xB has PPN 0x92 (valid)
• Page table at PPN 0x92, entry 0xE has PPN 0x55 (valid)
• Physical address: 0x55ED0

Lecture Note

Translating 0xBEED0 requires 2 memory accesses (one for the page directory, one for the inner page table).

Practice: Full System with TLBs

Assume a 3-level page table, 256-byte pages, 16-bit addresses, and ASID of current process is 211.

How many physical memory accesses for each instruction? (Ignore ops changing TLB)

Example

(a) 0xAA10: movl 0x1111, %edi — instruction fetch needs TLB lookup for VPN 0xAA (miss → 3 PT levels + 1 fetch = 4), data access needs TLB lookup for VPN 0x11 (miss → 3 PT levels + 1 access = 4). Total = 8

(b) 0xBB13: addl $0x3, %edi — instruction fetch: VPN 0xBB is a TLB hit (ASID 211, valid). Total = 1

(c) 0x0519: movl %edi, 0xFF10 — instruction fetch: VPN 0x05 has ASID 211 but valid=0 (miss → 3 + 1 = 4), data access: VPN 0xFF is a TLB hit. Total = 5

Large Pages

TLB reach: The amount of memory that can be accessed without a TLB miss.

With standard 4KB pages:

• 1000 TLB entries × 4KB = 4MB TLB reach

Most computers have far more memory than 4MB, so TLB misses are frequent.

Solution: Large pages

• 1000 TLB entries × 2MB pages = 2GB TLB reach

Page Tables for Large Pages

How do we translate large pages (2MB, 1GB) in a page table?

Make a page directory entry contain the address of the page frame directly, rather than a pointer to the next level.

x86-64 has 4 levels of page tables (9 bits each, 12-bit offset):

Lecture Note

2^9 = 512 entries per level. 512 × 4KB = 2MB (one page directory entry maps a 2MB large page). 512 × 2MB = 1GB (one page pointer directory entry can map a 1GB huge page).

• A PDE for a 2MB page contains the address of the 2MB page frame directly
• A PDE for a page table points to the next level
• This shortens the page walk by 1 or 2 levels

Using Large Pages

Programmer requested:

• mmap(MAP_HUGE) returns memory allocated with huge pages or fails
• Plus: control
• Minus: what do programmers really know?

Transparent Huge Pages (THP, in Linux):

• OS uses huge pages when available for > 2MB allocations
• Plus: automatic benefit
• Minus: creating huge pages can be slow, hard to know if worthwhile

Large vs Small Page Example

double sum = 0.0;

double a[1024*1024];
for (i=0; i<1024*1024; i++) {
    sum += a[rand() %
                (1024*1024)];
}

With 2MB pages (32 TLB entries):

• Array size: 8 × 1024 × 1024 = 8MB
• Number of large pages: 8MB / 2MB = 4 pages
• TLB lookups: 1M (number of accesses to array)
• TLB misses: 4 (number of unique pages accessed)
• Miss rate: 8/1,000,000
• TLB reach: 32 × 2MB = 64MB (covers entire array)

With 4KB pages (32 TLB entries):

• Number of small pages: 8MB / 4KB = 2048 pages
• TLB lookups: 1024 × 1024
• TLB misses: all of them (2048 pages >> 32 TLB entries)
• Miss rate: up to 100%
• Hit rate: 32/2048 ≈ 1.5%

User vs Kernel Addresses

With paging, the kernel uses virtual memory too.

• Low region of address space is private, per-process memory
• High region is reserved for kernel use and has the same translations for all processes
  • Privileged bit in PTE marks high region as only accessible when in privileged mode
  • All kernel data structures available when running any process
  • Kernel can only access usermode data of the running process

Lecture Note

Kernel region stores page tables (PT), process control blocks (PCB), and other per-process data. The privileged bit prevents user-mode access to the kernel region.

Virtual Address Space of a Linux Process

Kernel virtual memory (high addresses):

• Different for each process: Process-specific data structs (page tables, PCB, task and mm structs, kernel stack)
• Identical for each process: Physical memory mapping, kernel code and data

Lecture Note

“Heaps, stacks” — the kernel maintains its own heap and stacks for each process in the per-process kernel data.

Process virtual memory (low addresses, from high to low):

• User stack (%esp)
• Memory mapped region for shared libraries
• Runtime heap / malloc (brk)
• Data
• Code (starting at 0x00400000)

Shared Memory

Exploit the level of indirection between VA and PA:

• Regions of two separate processes’ address spaces map to the same physical pages
  • Read/write: access to share data
  • Execute: shared libraries
• Each process has separate PTEs, so processes can have different access privileges
• The shared region does not need to map to the same VA in each process

Lecture Note

No — shared memory regions do not need to be at the same virtual address in each process.

Where is page sharing used?

• Kernel data mapped into each process
• Different processes running the same binary
• User-level shared libraries
• Shared pages as IPC (inter-process communication)

Copy-on-Write

Copy-on-Write (COW): A technique used on fork() where parent and child initially share the same pages in memory. Pages are only copied when one process modifies a shared page.

On fork():

• Instead of copying all pages, create shared mappings of parent pages in child address space
  • Make shared mappings read-only in child space
  • When child does a write, a protection fault occurs, OS takes over and can then copy the page and resume

COW allows more efficient process creation as only modified pages are copied.

COW Mechanism

1. Setup: Two processes map a private COW object. Region is flagged as private copy-on-write. PTEs in private areas are flagged as read-only
2. On write: Instruction writing to a private page triggers a protection fault. Handler creates a new R/W page (copy of original). Instruction restarts upon handler return
3. Copying is deferred as long as possible

Swapping

Motivation

OS goal: Support processes when not enough physical memory

• Single process with very large address space
• Multiple processes with combined address spaces

User code should be independent of amount of physical memory

• Correctness, if not performance

Virtual memory: OS provides illusion of more physical memory

• Relies on key properties of user processes (workload) and machine architecture (hardware)

Demand Paging

Programs include many large libraries, some of which are rarely or never used. How to avoid wasting physical pages to back rarely used virtual pages?

Only keep the pages that are actively needed in physical memory. When a page is accessed that is not in memory, copy (or move) it to RAM. This is called “paging in.”

Locality of Reference

Leverage locality of reference within processes:

• Spatial: reference memory addresses near previously referenced addresses
• Temporal: reference memory addresses that have been referenced in the past
• Processes spend majority of time in small portion of code
  • Estimate: 90% of time in 10% of code

Implication:

• Process only uses a small amount of address space at any moment
• Only a small amount of address space must be resident in physical memory

Memory Hierarchy

Each layer acts as “backing store” for the layer above:

Virtual Address Space Mechanisms

Each page in virtual address space maps to one of three locations:

• Physical main memory: Small, fast, expensive → P=1, V=1
• Disk (backing store): Large, slow, cheap → P=0, V=1
• Nothing (error): Free → V=0

Extend page tables with an extra bit: present

• Permissions (r/w), valid, present
• Page in memory: present bit set in PTE
• Page on disk: present bit cleared
  • PTE points to block on disk
  • Causes trap into OS when page is referenced
  • Trap: page fault

Types of Page Faults

1. Segmentation fault — accessing a non-existing page (V=0)
2. Protection exception — e.g., violating permission by writing to a read-only page (Linux reports as segmentation fault)
3. Normal page fault — page is valid but not present (V=1, P=0), needs to be paged in from disk

Lecture Note

A protection exception may actually be a COW fault — the OS checks if the page is a copy-on-write page before killing the process.

Virtual Memory Mechanisms

First, hardware checks TLB for virtual address:

• If TLB hit, address translation is done; page is in physical memory → continue

Else (TLB miss):

• Hardware or OS walk page tables
• If PTE designates page is present, then page is in physical memory → add to TLB

Else (page fault):

• Trap into OS (not handled by hardware)
• OS finds an empty page
  • If none, OS selects victim page in memory to replace
  • Write victim page out to disk if modified (use dirty bit in PTE)
• OS reads referenced page from disk into memory
• Page table is updated, present bit is set
• Process continues execution

Lecture Note

On a page fault, a context switch occurs while the OS performs disk I/O to page in the needed data. The faulting process is blocked during this time.

Quiz Review

1. “Page table entries are stored in the PCB of a process when a context switch occurs.” → False (only the PT base pointer + ASID are saved)
2. “With a multi-level page table, the complete VPN is used as an index into the page directory.” → False (just the upper bits)
3. “A longer scheduling time slice is likely to decrease the overall TLB miss rate in the system.” → True
4. Consider 0x1acc: mov %ebx, [%esp] with a 2-level page table, empty TLB, and %ESP = 0x200. Memory references needed:
   • Instruction fetch: 2 for PT walk + 1 fetch = 3
   • Data access: 2 for PT walk + 1 for mov = 3
   • Total = 6 memory references

Page Table Size Comparison

Consider a system with 32-bit virtual addresses, 4KB pages, and 4-byte PTEs. Process has: Code 1MB, Global data 4KB, Heap 20MB, Stack 64KB.

• Linear page table: 2^20 entries × 4 bytes = 4MB (of mappings, regardless of what’s used)
• 2-level page table (10-bit outer, 10-bit inner): 1 outer page directory + 1 inner PT for stack + 6 inner PTs for code/data/heap = 8 × 4KB = 32KB

Lecture Note

The 2-level page table only allocates inner page tables for address ranges that are actually in use, saving significant memory.

Swapping Policies

Goal: Minimize number of page faults

• Page faults require milliseconds to handle (reading from disk)
• Implication: Plenty of time for OS to make a good decision

OS has two decisions:

• Page selection: When should a page on disk be brought into memory?
• Page replacement: Which resident page in memory should be thrown out to disk?

Page Selection

Demand paging: Load page only when page fault occurs

• Intuition: Wait until page must absolutely be in memory
• When process starts: No pages are loaded in memory
• Problems: Pay cost of page fault for every newly accessed page

Prepaging (anticipatory, prefetching): Load page before referenced

• OS predicts future accesses and brings pages into memory early
• Works well for some access patterns (e.g., sequential)

Hints: Combine above with user-supplied hints about page references

• User specifies: may need page in future, don’t need this page anymore, or sequential access pattern
• Example: madvise() in Linux

Page Replacement

Which page in main memory should be selected as victim?

• Write out victim page to disk if modified (dirty bit set)
• If victim page is not modified (clean), just discard

OPT: Replace page not used for longest time in future

• Advantages: Guaranteed to minimize number of page faults
• Disadvantages: Requires that OS predict the future; not practical, but good for comparison

FIFO: Replace page that has been in memory the longest

• Intuition: First referenced long time ago, done with it now
• Advantages: Fair (all pages receive equal residency); easy to implement
• Disadvantage: Some pages may always be needed

LRU: Replace page not used for longest time in past

• Intuition: Use past to predict the future
• Advantages: With temporal locality, LRU approximates OPT
• Disadvantages:
  • Harder to implement, must track which pages have been accessed
  • Does not handle all workloads well

Page Replacement Example

Page reference string: A B C A B D A D B C B (3 pages of physical memory)

Miss count: OPT = 5, FIFO = 7, LRU = 5

Page Replacement Comparison

Add more physical memory — what happens to performance?

LRU, OPT:

• Guaranteed to have fewer (or same number of) page faults
• Smaller memory sizes are guaranteed to contain a subset of larger memory sizes
• Stack property: smaller cache always subset of bigger

FIFO:

• Usually has fewer page faults
• Belady’s anomaly: May actually have more page faults with more memory!

Belady’s Anomaly (FIFO)

Page requests: C B A E C B D C B A E D

More pages (4) → more misses (10) than fewer pages (3, 9 misses)!

Implementing LRU

Software Perfect LRU:

• OS maintains ordered list of physical pages by reference time
• When page is referenced: Move page to front of list
• When need victim: Pick page at back of list
• Trade-off: Slow on memory reference, fast on replacement

Hardware Perfect LRU:

• Associate timestamp register with each page
• When page is referenced: Store system clock in register
• When need victim: Scan through registers to find oldest clock
• Trade-off: Fast on memory reference, slow on replacement (especially as size of memory grows)

In practice, do not implement Perfect LRU:

• LRU is an approximation anyway, so approximate more
• Goal: Find an old page, but not necessarily the very oldest

Clock Algorithm

Hardware:

• Keep use (or reference) bit for each page frame
• When page is referenced: set use bit

Operating System:

• Page replacement: Look for page with use bit cleared (has not been referenced for a while)
• Implementation:
  • Keep pointer to last examined page frame
  • Traverse pages in circular buffer
  • Clear use bits as you search
  • Stop when you find page with already-cleared use bit — replace this page

The single clock hand:

• Advances only on page fault
• Checks for pages not used recently
• Marks pages as not used recently

What does the clock hand speed tell us?

• Hand moving slowly → good sign: not many page faults and/or finding pages quickly
• Hand moving quickly → bad sign: lots of page faults and/or lots of reference bits set

The clock algorithm is a crude partitioning of pages into two groups: young and old (Second Chance — page wasn’t used during last cycle and current cycle).

Clock Walkthrough

Physical memory: pages [0, 1, 2, 3, …], clock hand starts at page 0.

Initial state: use=1, use=1, use=0, use=1

Page fault occurs: Need to find a victim.

1. Page 0: use=1 → clear to 0, advance hand
2. Page 1: use=1 → clear to 0, advance hand
3. Page 2: use=0 → evict page 2

Later, page 0 is accessed → use bit for page 0 set back to 1.

Next page fault: hand is at page 2.

1. Page 2: use=0 (just loaded, depends on access) → advance
2. Page 3: use=1 → clear to 0, advance, wraps around
3. Page 0: use=1 → clear to 0, advance
4. Page 1: use=0 → evict page 1

Clock Extensions

Batch eviction:

• Replace multiple pages at once
• Intuition: Expensive to run replacement algorithm and to write single block to disk
• Find multiple victims each time and track free list

Dirty bit preference:

• Use dirty bit to give preference to clean pages
• Intuition: More expensive to replace dirty pages (dirty pages must be written to disk, clean pages do not)
• Replace pages that have use bit and dirty bit cleared

Second Chance Page Replacement

LRU Clock is a global algorithm:

• It looks at all physical pages, from all processes
• Every process gets its memory taken away gradually

Second chance FIFO (per-process):

• Maintain 2 FIFO queues per process: active and inactive
  • Fixed proportion (e.g. 50:50, 75:25)
• On first access, pages go at tail of active queue, pushes off head
• When pushed off active queue, move to inactive queue and clear present bit P
• When they drop off inactive queue, they are replaced
• If they are accessed in inactive queue, they are put back on active queue

Second Chance Walkthrough

Initial state: Active = [9, 1, 3, 6] (tail→head), Inactive = [4, 5, 8, 7] (tail→head). Page 7 is being evicted to disk.

Reference A (new page): A enters tail of active, pushes 6 off head → Active = [A, 9, 1, 3], Inactive = [6, 4, 5, 8]. Page 7 evicted to disk.

Reference 5 (in inactive list — fault!): Page 5 found in inactive list → move back to active tail. Active = [5, A, 9, 1], pushes 3 off head → Inactive = [3, 6, 4, 8]. Page 7 already on disk.

Reference D (new page, not in either list — fault!): D enters tail of active, pushes 1 off head → Active = [D, 5, A, 9], Inactive = [1, 3, 6, 4]. Pages 7, 8 on disk.

Advantages: No scanning — always evict inactive head. Disadvantage: Extra soft page faults (when accessing pages in the inactive list).

Frame Allocation

In a multiprogramming system, we need a way to allocate physical memory to competing processes.

Fixed space algorithms:

• Each process is given a limit of pages it can use
• When it reaches its limit, it replaces from its own pages
• Local replacement: some processes may do well, others suffer

Variable space algorithms:

• Processes’ set of pages grows and shrinks dynamically
• Global replacement: one process can ruin it for the rest
  • Clock is global replacement

Working Set

How much memory does a process need?

Working set W(D,t): The set of pages that have been referenced in the last D virtual time units.

• Virtual time = time elapsed while the process was in execution (e.g., number of instructions executed)
• D is a window of time
• At any t, |W(D,t)| is non-decreasing with D
• W(D,t) is an approximation of the program’s locality

The Working Set Strategy

The working set of a process:

1. First grows when it starts executing
2. Then stabilizes by the principle of locality
3. Grows again when the process enters a new locality (transition period)
   • Working set contains pages from two localities
4. Then decreases after a sufficient long time spent in the new locality

How to determine sizes:

• Monitor page fault rate
• High rate → needs more memory
• Low rate → has enough

Working set model assumes locality — principle of locality: a program clusters access to data and text temporally. As the number of physical pages exceeds the working set size, page fault rate drops dramatically.

Thrashing

Computation has locality. As page frames decrease, the page frames available are not large enough to contain the locality of the process.

The processes start faulting heavily. Pages that are read in are used, then immediately paged out.

Thrashing and CPU utilization:

• As page fault rate goes up, processes get suspended on page out queues for the disk
• System may try to optimize performance by starting new jobs
• Starting new jobs will reduce the number of page frames available to each process, increasing the page fault requests
• System throughput plunges

Solution? Stop running some processes.

Swap Storage

Where Do Pages Go on Disk?

Kernel organizes regions of virtual memory as “areas” or segments according to how they are swapped:

• Data that gets swapped to the same file all goes to a segment
• Multiple memory areas can get swapped to the same file in different places, or to anywhere in a “swap file” or “swap partition”

How do you find a place to swap a page in the swap file?

• Swap daemon maintains a swap map of:
  • Which blocks on disk are in use
  • Which virtual pages are stored in those blocks

When Is a Page Allocated Space?

• At allocation time:
  • Ensures space available
  • Saves time on swapping
  • Total memory usage = swap
• At eviction time:
  • May never need to do it
  • Can put it in a better place, so write pages sequentially
  • Total memory usage = swap + RAM

Memory-Mapped Files

Memory-mapped file I/O allows file I/O to be treated as routine memory access by mapping a disk block to a page in memory.

A file is initially read using demand paging. A page-sized portion of the file is read from the file system into a physical page. Subsequent reads/writes to/from the file are treated as ordinary memory accesses.

Simplifies file access by treating file I/O through memory rather than read() / write() system calls.

Also allows several processes to map the same file, allowing the pages in memory to be shared.

How mmap() works:

1. mmap() maps file to a region of the virtual address space
2. PT entries for mapped region are marked as “backed” by file
3. On access, page fault occurs → page fault handler reads file contents into a physical page
4. Subsequent accesses read file contents from memory
5. Scheduler may context-switch during the disk I/O

Summary: Virtual Memory

• Abstraction: Virtual address space with code, heap, stack
• Address translation:
  • Contiguous memory: base, bounds, segmentation
  • Using fixed-size pages with page tables
• Challenges with paging:
  • Extra memory references: avoid with TLB
  • Page table size: avoid with multi-level paging, inverted page tables, etc.
• Larger address spaces: Swapping mechanisms, policies (LRU, Clock)

Related

• Navigation: ← Lecture 7 · Lecture 9 →
• Exam: Midterm 1
• Course: CS 537 Map of Content