Lecture 9 - Memory: Page Replacement

Instructor: Michael Swift Topics: Demand paging, page replacement policies, clock algorithm, working set, thrashing, memory-mapped files

Learning Outcomes

• Understand how to use page tables to share data efficiently
• Understand how the kernel accesses user data
• Understand how swapping works
• Apply swapping policies to determine the number of page faults

Recap: Present Bit

Each page in virtual address space maps to one of three locations:

• Physical main memory: Small, fast, expensive
• Disk (backing store): Large, slow, cheap
• Nothing (error): Free

Extend page tables with an extra bit: present

• Permissions (r/w), valid, present
• Page in memory: present bit set in PTE
• Page on disk: present bit cleared
  • PTE points to block on disk
  • Causes trap into OS when page is referenced
  • Trap: page fault

Recap: Page Fault Handling

Page Fault Handling Steps:

1. Process references a page (load M)
2. Reference traps to the operating system
3. OS finds the page on backing store (disk)
4. OS brings the missing page into a free frame in physical memory
5. OS resets the page table (sets present bit)
6. Process restarts the instruction

Swapping Policies

Goal: Minimize number of page faults

• Page faults require milliseconds to handle (reading from disk)
• Implication: Plenty of time for OS to make good decision

OS has two decisions:

• Page selection — When should a page (or pages) on disk be brought into memory?
• Page replacement — Which resident page (or pages) in memory should be thrown out to disk?

Page Selection

Demand paging: Load page only when page fault occurs

• Intuition: Wait until page must absolutely be in memory
• When process starts: No pages are loaded in memory
• Problems: Pay cost of page fault for every newly accessed page

Prepaging (anticipatory, prefetching): Load page before referenced

• OS predicts future accesses and brings pages into memory early
• Works well for some access patterns (e.g., sequential)

Hints: Combine above with user-supplied hints about page references

• User specifies: may need page in future, don’t need this page anymore, or sequential access pattern, …
• Example: madvise() in Linux

Page Replacement

Which page in main memory should be selected as victim?

• Write out victim page to disk if modified (dirty bit set)
• If victim page is not modified (clean), just discard

What is the best possible choice?

OPT: Replace page not used for longest time in future

• Advantages: Guaranteed to minimize number of page faults
• Disadvantages: Requires that OS predict the future; not practical, but good for comparison

Page Replacement Policies

FIFO

FIFO: Replace page that has been in memory the longest

• Intuition: First referenced long time ago, done with it now
• Advantages: Fair — all pages receive equal residency; easy to implement
• Disadvantage: Some pages may always be needed

LRU

LRU: Least-recently-used — replace page not used for longest time in past

• Intuition: Use past to predict the future
• Advantages: With temporal locality, LRU approximates OPT
• Disadvantages:
  • Harder to implement, must track which pages have been accessed
  • Does not handle all workloads well (e.g., sequential)

Lecture Note

For sequential workloads, random replacement can actually perform better than LRU.

Page Replacement Example

Page reference string: D D B B A C B D B D Three pages of physical memory. Metric: miss count.

• OPT: 4 faults
• FIFO: 6 faults
• LRU: 5 faults

Page Replacement Comparison

Add more physical memory — what happens to performance?

LRU, OPT:

• Guaranteed to have fewer (or same number of) page faults
• Smaller memory sizes are guaranteed to contain a subset of larger memory sizes
• Stack property: smaller cache always subset of bigger

FIFO:

• Usually has fewer page faults
• Belady’s anomaly: May actually have more page faults with more memory!

Belady’s Anomaly Example

Page reference string: C B A E C B D C B A E D

Italics indicate page faults. More memory (4 pages) leads to more faults (10) than fewer memory (3 pages, 9 faults).

Implementing LRU

Software Perfect LRU

• OS maintains ordered list of physical pages by reference time
• When page is referenced: Move page to front of list
• When need victim: Pick page at back of list
• Trade-off: Slow on memory reference, fast on replacement

Hardware Perfect LRU

• Associate timestamp register with each page
• When page is referenced: Store system clock in register
• When need victim: Scan through registers to find oldest clock
• Trade-off: Fast on memory reference, slow on replacement (especially as size of memory grows)

Lecture Note

In practice, do not implement perfect LRU. LRU is an approximation anyway, so approximate more. Goal: Find an old page, but not necessarily the very oldest — Not Recently Used.

Clock Algorithm

Hardware Support

• Keep use (or reference) bit for each page frame
• When page is referenced: set use bit

Operating System

• Page replacement: Look for page with use bit cleared (has not been referenced for a while)
• Implementation:
  • Keep pointer to last examined page frame
  • Traverse pages in circular buffer
  • Clear use bits as search proceeds
  • Stop when find page with already cleared use bit — replace this page

How the Clock Hand Works

The single clock hand:

• Advances only on page fault
• Checks for pages not used recently
• Marks pages as not used recently

Example

Physical memory: pages 0, 1, 2, 3, …

Initial state: use bits = [1, 1, 0, 1], clock hand at page 0

1. Page 0: use=1 → clear to 0, advance hand
2. Page 1: use=1 → clear to 0, advance hand
3. Page 2: use=0 → evict page 2

Later, page 0 is accessed → hardware sets use=1 for page 0.

Next page fault: hand is at page 2, starts scanning again. Eventually finds page 1 (use=0) → evict page 1.

Clock Speed Interpretation

• Hand moving slowly — good sign: not many page faults and/or finding a page to evict quickly
• Hand moving quickly — bad sign: lots of page faults and/or lots of reference bits set

One way to view the clock algorithm:

• Crude partitioning of pages into two groups: young and old

Second Chance Improvement

• Page wasn’t used during last cycle and current cycle

Clock Extensions

Replace multiple pages at once:

• Intuition: Expensive to run replacement algorithm and to write single block to disk
• Find multiple victims each time and track free list

Use dirty bit to give preference to clean pages:

• Intuition: More expensive to replace dirty pages — dirty pages must be written to disk, clean pages do not
• Replace pages that have use bit and dirty bit cleared

Frame Allocation

In a multiprogramming system, we need a way to allocate physical memory to competing processes:

• What if a victim page belongs to another process?
• Family of replacement algorithms that takes this into account

Fixed Space Algorithms

• Each process is given a limit of pages it can use
• When it reaches its limit, it replaces from its own pages
• Local replacement: some processes may do well, others suffer

Variable Space Algorithms

• Processes’ set of pages grows and shrinks dynamically
• Global replacement: one process can ruin it for the rest

Second Chance Page Replacement

LRU Clock is a global algorithm:

• It looks at all physical pages, from all processes
• Every process gets its memory taken away gradually

Second Chance FIFO

• Maintain 2 FIFO queues per process: active, inactive
  • Fixed proportion (e.g., 50:50, 75:25)
• On first access, pages go at tail of active queue, pushes off head
• When pushed off active queue, move to inactive queue and clear present bit
• When they drop off inactive queue, they are replaced
• If they are accessed in inactive queue, they are put back on active queue

Example

Reference A: Active list [9, 1, 3, 6] (present=1), Inactive list [4, 5, 8, 7] (present=0). Page 7 drops off inactive → swapped to disk.

Reference 5: Page 5 is in inactive list → page fault (soft), move 5 to active list tail. Page A goes to tail of active, pushes head (6) to inactive. Active: [A, 9, 1, 3], Inactive: [6, 4, 5, 8]. Page 7 already on disk.

Reference D: Page D is not in memory → page fault. Evict head of inactive (page 8). Active: [D, 5, A, 9], Inactive: [1, 3, 6, 4].

Advantages:

• No scanning — always evict inactive head
• Extra soft page faults (page still in memory, just on inactive list)

Working Set

What is the right amount of memory to give a process?

• All memory allocated?
• What it is using this cycle?

Working Set: The working set for a process at time t, W(D,t), is the set of pages that have been referenced in the last D virtual time units.

• Virtual time = time elapsed while the process was in execution (e.g., number of instructions executed)
• D is a window of time
• At any t, |W(D,t)| is non-decreasing with D
• W(D,t) is an approximation of the program’s locality

Working Set Model

Working set model assumes locality.

Principle of locality: a program clusters access to data and text temporally.

As the number of physical pages exceeds a threshold (the working set size), page fault rate drops dramatically.

The Working Set Strategy

The working set of a process:

1. First grows when it starts executing
2. Then stabilizes by the principle of locality
3. Grows again when the process enters a new locality (transition period)
   • Up to a point where the working set contains pages from two localities
4. Then decreases after a sufficient long time spent in the new locality

How to determine sizes:

• Monitor page fault rate
• High rate → needs more memory
• Low rate → has enough

Thrashing

Thrashing: When page frames available are not large enough to contain the locality of the process, causing heavy page faulting where pages that are read in are used and immediately paged out.

• As page fault rate goes up, processes get suspended on page out queues for the disk
• System may try to optimize performance by starting new jobs
• Starting new jobs will reduce the number of page frames available to each process, increasing the page fault requests
• System throughput plunges

Key Insight

Solution to thrashing: stop running some processes to free up memory for the remaining ones.

Where Do Pages Go on Disk?

Kernel organizes regions of virtual memory as “areas” or segments according to how they are swapped:

• Data that gets swapped to the same file all goes to a segment
• Multiple memory areas can get swapped to the same file in different places, or to anywhere in a “swap file” or “swap partition”

How do you find a place to swap a page in the swap file?

• Swap daemon maintains a swap map of:
  • Which blocks on disk are in use
  • Which virtual pages are stored in those blocks

Memory Allocation

Option 1: Grow the heap

• brk() — set end of heap
• sbrk() — adjusts end of heap
• Downside: can’t free memory in middle

Option 2: Create new regions

• mmap(MAP_ANONYMOUS) — create new memory region
• Used by malloc() for new regions
• munmap(vaddr, len) — frees range of addresses

Memory-Mapped Files

Memory-mapped file I/O allows file I/O to be treated as routine memory access by mapping a disk block to a page in memory.

A file is initially read using demand paging. A page-sized portion of the file is read from the file system into a physical page. Subsequent reads/writes to/from the file are treated as ordinary memory accesses.

Simplifies file access by treating file I/O through memory rather than read() / write() system calls.

Also allows several processes to map the same file, allowing the pages in memory to be shared.

How mmap Works

1. Process calls mmap() to map a file to a region of its virtual address space
2. OS creates page table entries for the mapped region as “backed” by the file
3. On first access, a page fault occurs
4. Page fault handler reads the page-sized portion of the file from disk into a physical frame
5. Page table is updated, process retries the instruction
6. Subsequent accesses to that page hit physical memory directly

Sharing Through Mapped Files

Multiple processes can map the same file into their respective virtual address spaces. The OS maps both virtual address regions to the same physical pages in memory, enabling shared access to the file data.

Summary: Virtual Memory

Abstraction: Virtual address space with code, heap, stack

Address translation:

• Contiguous memory: base, bounds, segmentation
• Using fixed-size pages with page tables

Challenges with paging:

• Extra memory references: avoid with TLB
• Page table size: avoid with multi-level paging, inverted page tables, etc.

Larger address spaces: Swapping mechanisms, policies (LRU, Clock)

Related

• Navigation: ← Lecture 8 · Lecture 10 →
• Exam: Midterm 1
• Course: CS 537 Map of Content