# Lecture 9 - Memory: Page Replacement

Source: https://wisconsin.twango.dev/sp26-cs537/lectures/lecture-09

# Lecture 9 - Memory: Page Replacement

**Instructor:** Michael Swift
**Topics:** Demand paging, page replacement policies, clock algorithm, working set, thrashing, memory-mapped files

> [!abstract]- Learning Outcomes
> - Understand how to use page tables to share data efficiently
> - Understand how the kernel accesses user data
> - Understand how swapping works
> - Apply swapping policies to determine the number of page faults

---

## Recap: Present Bit

Each page in virtual address space maps to one of three locations:
- **Physical main memory:** Small, fast, expensive
- **Disk (backing store):** Large, slow, cheap
- **Nothing (error):** Free

Extend page tables with an extra bit: **present**
- Permissions (r/w), valid, present
- Page in memory: present bit set in PTE
- Page on disk: present bit cleared
  - PTE points to block on disk
  - Causes trap into OS when page is referenced
  - Trap: **page fault**

---

## Recap: Page Fault Handling

> **Page Fault Handling Steps:**
>
> 1. Process references a page (`load M`)
> 2. Reference traps to the operating system
> 3. OS finds the page on backing store (disk)
> 4. OS brings the missing page into a free frame in physical memory
> 5. OS resets the page table (sets present bit)
> 6. Process restarts the instruction

---

## Swapping Policies

**Goal:** Minimize number of page faults
- Page faults require milliseconds to handle (reading from disk)
- Implication: Plenty of time for OS to make good decision

OS has two decisions:
- **Page selection** — **When** should a page (or pages) on disk be **brought into** memory?
- **Page replacement** — **Which** resident page (or pages) in memory should be **thrown out** to disk?

---

## Page Selection

**Demand paging:** Load page only when page fault occurs
- Intuition: Wait until page must absolutely be in memory
- When process starts: No pages are loaded in memory
- Problems: Pay cost of page fault for every newly accessed page

**Prepaging (anticipatory, prefetching):** Load page before referenced
- OS predicts future accesses and brings pages into memory early
- Works well for some access patterns (e.g., sequential)

**Hints:** Combine above with user-supplied hints about page references
- User specifies: may need page in future, don't need this page anymore, or sequential access pattern, ...
- Example: `madvise()` in Linux

---

## Page Replacement

Which page in main memory should be selected as victim?
- Write out victim page to disk if modified (dirty bit set)
- If victim page is not modified (clean), just discard

What is the **best possible** choice?

**OPT:** Replace page not used for longest time in future
- Advantages: Guaranteed to minimize number of page faults
- Disadvantages: Requires that OS predict the future; not practical, but good for comparison

---

## Page Replacement Policies

### FIFO

**FIFO:** Replace page that has been in memory the longest
- Intuition: First referenced long time ago, done with it now
- Advantages: Fair — all pages receive equal residency; easy to implement
- Disadvantage: Some pages may always be needed

### LRU

**LRU:** Least-recently-used — replace page not used for longest time in past
- Intuition: Use past to predict the future
- Advantages: With temporal locality, LRU approximates OPT
- Disadvantages:
  - Harder to implement, must track which pages have been accessed
  - Does not handle all workloads well (e.g., sequential)

> [!note] Lecture Note
> For sequential workloads, random replacement can actually perform better than LRU.

---

## Page Replacement Example

Page reference string: **D D B B A C B D B D**
Three pages of physical memory. Metric: miss count.

| Reference | OPT | FIFO | LRU |
|-----------|-----|------|-----|
| D | **D** _ _ (miss) | **D** _ _ (miss) | **D** _ _ (miss) |
| D | D _ _ | D _ _ | D _ _ |
| B | D **B** _ (miss) | D **B** _ (miss) | D **B** _ (miss) |
| B | D B _ | D B _ | D B _ |
| A | D B **A** (miss) | D B **A** (miss) | D B **A** (miss) |
| C | D B **C** (miss) | **C** B A (miss) | **C** B A (miss) |
| B | D B C | C B A | C B A |
| D | D B C | C **D** A (miss) | C B **D** (miss) |
| B | D B C | C D **B** (miss) | C B D |
| D | D B C | C D B | C B D |

- **OPT:** 4 faults
- **FIFO:** 6 faults
- **LRU:** 5 faults

---

## Page Replacement Comparison

Add more physical memory — what happens to performance?

**LRU, OPT:**
- Guaranteed to have fewer (or same number of) page faults
- Smaller memory sizes are guaranteed to contain a subset of larger memory sizes
- **Stack property:** smaller cache always subset of bigger

**FIFO:**
- Usually has fewer page faults
- **Belady's anomaly:** May actually have **more** page faults with more memory!

### Belady's Anomaly Example

Page reference string: **C B A E C B D C B A E D**

| | C | B | A | E | C | B | D | C | B | A | E | D |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| **3 pages** | *C* | *B* | *A* | *E* | *C* | *B* | D | D | D | *A* | *E* | E |
| | | C | B | A | E | C | B | B | B | D | A | A |
| | | | C | B | A | E | C | C | C | B | D | D |
| **Result** | | | | | | | | | | **9 misses** | | |

| | C | B | A | E | C | B | D | C | B | A | E | D |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| **4 pages** | *C* | *B* | *A* | *E* | E | E | *D* | *C* | *B* | *A* | *E* | *D* |
| | | C | B | A | A | A | E | D | C | B | A | E |
| | | | C | B | B | B | A | E | D | C | B | A |
| | | | | C | C | C | B | A | E | D | C | B |
| **Result** | | | | | | | | | | **10 misses** | | |

*Italics* indicate page faults. More memory (4 pages) leads to more faults (10) than fewer memory (3 pages, 9 faults).

---

## Implementing LRU

### Software Perfect LRU

- OS maintains ordered list of physical pages by reference time
- When page is referenced: Move page to front of list
- When need victim: Pick page at back of list
- Trade-off: Slow on memory reference, fast on replacement

### Hardware Perfect LRU

- Associate timestamp register with each page
- When page is referenced: Store system clock in register
- When need victim: Scan through registers to find oldest clock
- Trade-off: Fast on memory reference, slow on replacement (especially as size of memory grows)

> [!note] Lecture Note
> In practice, do not implement perfect LRU. LRU is an approximation anyway, so approximate more. Goal: Find an old page, but not necessarily the very oldest — **Not Recently Used**.

---

## Clock Algorithm

### Hardware Support

- Keep **use** (or reference) bit for each page frame
- When page is referenced: set use bit

### Operating System

- Page replacement: Look for page with use bit cleared (has not been referenced for a while)
- Implementation:
  - Keep pointer to last examined page frame
  - Traverse pages in circular buffer
  - Clear use bits as search proceeds
  - Stop when find page with already cleared use bit — replace this page

### How the Clock Hand Works

| Use Bit | Action |
|---------|--------|
| 0 | Evict the page |
| 1 | Clear use bit and advance hand |

The single clock hand:
- Advances **only on page fault**
- Checks for pages not used recently
- Marks pages as not used recently

> [!example] Example
> Physical memory: pages 0, 1, 2, 3, ...
>
> **Initial state:** use bits = [1, 1, 0, 1], clock hand at page 0
>
> 1. Page 0: use=1 → clear to 0, advance hand
> 2. Page 1: use=1 → clear to 0, advance hand
> 3. Page 2: use=0 → **evict page 2**
>
> Later, page 0 is accessed → hardware sets use=1 for page 0.
>
> **Next page fault:** hand is at page 2, starts scanning again.
> Eventually finds page 1 (use=0) → **evict page 1**.

### Clock Speed Interpretation

- **Hand moving slowly** — good sign: not many page faults and/or finding a page to evict quickly
- **Hand moving quickly** — bad sign: lots of page faults and/or lots of reference bits set

One way to view the clock algorithm:
- Crude partitioning of pages into two groups: **young** and **old**

### Second Chance Improvement

- Page wasn't used during last cycle *and* current cycle

---

## Clock Extensions

**Replace multiple pages at once:**
- Intuition: Expensive to run replacement algorithm and to write single block to disk
- Find multiple victims each time and track free list

**Use dirty bit to give preference to clean pages:**
- Intuition: More expensive to replace dirty pages — dirty pages must be written to disk, clean pages do not
- Replace pages that have use bit **and** dirty bit cleared

---

## Frame Allocation

In a multiprogramming system, we need a way to allocate physical memory to competing processes:
- What if a victim page belongs to another process?
- Family of replacement algorithms that takes this into account

### Fixed Space Algorithms

- Each process is given a limit of pages it can use
- When it reaches its limit, it replaces from its own pages
- **Local replacement:** some processes may do well, others suffer

### Variable Space Algorithms

- Processes' set of pages grows and shrinks dynamically
- **Global replacement:** one process can ruin it for the rest

---

## Second Chance Page Replacement

LRU Clock is a **global** algorithm:
- It looks at all physical pages, from all processes
- Every process gets its memory taken away gradually

### Second Chance FIFO

- Maintain 2 FIFO queues per process: **active**, **inactive**
  - Fixed proportion (e.g., 50:50, 75:25)
- On first access, pages go at tail of active queue, pushes off head
- When pushed off active queue, move to inactive queue and clear present bit
- When they drop off inactive queue, they are replaced
- If they are accessed in inactive queue, they are put back on active queue

> [!example] Example
> **Reference A:** Active list [9, 1, 3, 6] (present=1), Inactive list [4, 5, 8, 7] (present=0). Page 7 drops off inactive → swapped to disk.
>
> **Reference 5:** Page 5 is in inactive list → page fault (soft), move 5 to active list tail. Page A goes to tail of active, pushes head (6) to inactive. Active: [A, 9, 1, 3], Inactive: [6, 4, 5, 8]. Page 7 already on disk.
>
> **Reference D:** Page D is not in memory → page fault. Evict head of inactive (page 8). Active: [D, 5, A, 9], Inactive: [1, 3, 6, 4].

Advantages:
- No scanning — always evict inactive head
- Extra **soft** page faults (page still in memory, just on inactive list)

---

## Working Set

What is the right amount of memory to give a process?
- All memory allocated?
- What it is using this cycle?

> **Working Set:** The working set for a process at time t, W(D,t), is the set of pages that have been referenced in the last D virtual time units.

- **Virtual time** = time elapsed while the process was in execution (e.g., number of instructions executed)
- D is a window of time
- At any t, |W(D,t)| is non-decreasing with D
- W(D,t) is an approximation of the program's locality

### Working Set Model

Working set model assumes **locality**.

**Principle of locality:** a program clusters access to data and text temporally.

As the number of physical pages exceeds a threshold (the working set size), page fault rate drops dramatically.

### The Working Set Strategy

The working set of a process:
1. First grows when it starts executing
2. Then stabilizes by the principle of locality
3. Grows again when the process enters a new locality (transition period)
   - Up to a point where the working set contains pages from two localities
4. Then decreases after a sufficient long time spent in the new locality

How to determine sizes:
- Monitor page fault rate
- High rate → needs more memory
- Low rate → has enough

---

## Thrashing

> **Thrashing:** When page frames available are not large enough to contain the locality of the process, causing heavy page faulting where pages that are read in are used and immediately paged out.

- As page fault rate goes up, processes get suspended on page out queues for the disk
- System may try to optimize performance by starting new jobs
- Starting new jobs will reduce the number of page frames available to each process, increasing the page fault requests
- System throughput plunges

> [!tip] Key Insight
> Solution to thrashing: stop running some processes to free up memory for the remaining ones.

---

## Where Do Pages Go on Disk?

Kernel organizes regions of virtual memory as "areas" or segments according to how they are swapped:
- Data that gets swapped to the same file all goes to a segment
- Multiple memory areas can get swapped to the same file in different places, or to anywhere in a "swap file" or "swap partition"

How do you find a place to swap a page in the swap file?
- Swap daemon maintains a **swap map** of:
  - Which blocks on disk are in use
  - Which virtual pages are stored in those blocks

---

## Memory Allocation

**Option 1: Grow the heap**
- `brk()` — set end of heap
- `sbrk()` — adjusts end of heap
- Downside: can't free memory in middle

**Option 2: Create new regions**
- `mmap(MAP_ANONYMOUS)` — create new memory region
- Used by `malloc()` for new regions
- `munmap(vaddr, len)` — frees range of addresses

---

## Memory-Mapped Files

Memory-mapped file I/O allows file I/O to be treated as routine memory access by **mapping** a disk block to a page in memory.

A file is initially read using demand paging. A page-sized portion of the file is read from the file system into a physical page. Subsequent reads/writes to/from the file are treated as ordinary memory accesses.

Simplifies file access by treating file I/O through memory rather than `read()` / `write()` system calls.

Also allows several processes to map the same file, allowing the pages in memory to be shared.

### How mmap Works

1. Process calls `mmap()` to map a file to a region of its virtual address space
2. OS creates page table entries for the mapped region as "backed" by the file
3. On first access, a page fault occurs
4. Page fault handler reads the page-sized portion of the file from disk into a physical frame
5. Page table is updated, process retries the instruction
6. Subsequent accesses to that page hit physical memory directly

### Sharing Through Mapped Files

Multiple processes can map the same file into their respective virtual address spaces. The OS maps both virtual address regions to the same physical pages in memory, enabling shared access to the file data.

---

## Summary: Virtual Memory

**Abstraction:** Virtual address space with code, heap, stack

**Address translation:**
- Contiguous memory: base, bounds, segmentation
- Using fixed-size pages with page tables

**Challenges with paging:**
- Extra memory references: avoid with TLB
- Page table size: avoid with multi-level paging, inverted page tables, etc.

**Larger address spaces:** Swapping mechanisms, policies (LRU, Clock)

---

## Related

- **Navigation:** [[lecture-08|← Lecture 8]] · [[lecture-10|Lecture 10 →]]
- **Exam:** [[exams/midterm-1/index|Midterm 1]]
- **Course:** [[README|CS 537 Map of Content]]