# Lecture 4 - Advanced Scheduling

Source: https://wisconsin.twango.dev/sp26-cs537/lectures/lecture-04

# Lecture 4 - Advanced Scheduling

**Instructor:** Tej Chajed
**Topics:** Proportional share scheduling, CFS, weighted fair scheduling, multiprocessor scheduling

> [!abstract]- Learning Outcomes
> - Compare and contrast basic scheduling (e.g., MLFQ) with fair scheduling (e.g., CFS)
> - Analyze limitations of fair scheduling
> - Articulate the challenges of multiprocessor scheduling

---

## Proportional Share Scheduling

Goal: give each of N processes **1/Nth** of the time on a CPU

- 1 process: 100% of CPU
- 3 processes: each get 1/3rd of CPU

Answers **how much** time a process gets but not **when** a process runs.

**Fairness:** each process gets the same amount — contrast with MLFQ's goals.

---

## Completely Fair Scheduler (CFS)

### Concept

Idea: give each of N processes **1/Nth** of the time on a CPU

- Consider ideal multitasking CPU with infinitesimally small timeslice
  - Context switch instantly after each instruction
- N processes: each progresses uniformly at **1/Nth** rate

> [!note] Lecture Note
> With 1 process, it gets 100% progression speed. With 3 processes, each makes 1/3rd progress.

### Design

Goal: give each of N processes **1/Nth** of the time on a CPU

How (generally):
- Keep track of CPU time usage of a process (*runtime*)
- Always run the process with the lowest runtime

How it works:
- I/O bound processes haven't run in a while, so they are behind and get to catch up quickly
  - **Low response time**
- CPU bound processes have run recently, so they get preempted

### Fair Scheduling Example

| Job | Arrival(s) | Run time (s) | Completion time |
|-----|------------|---------------|-----------------|
| A   | 0          | 50            | 85              |
| B   | 10         | 15            | 40              |
| C   | 15         | 10            | 30              |
| B   | 50         | 10            | 60              |

> [!example] Example
> At time 15, runtime values are A: 10, B: 5, C: 0. C has the lowest runtime so it runs next. Between times 20-30, A, B, and C all have similar runtimes and rapidly alternate. The scheduler always picks whichever process has the lowest runtime value.

**Average Turnaround Time:** (85-0) + (40-10) + (30-15) + (60-50) / 4 = **35**

**Average Response Time:** **0** (every job runs immediately upon arrival since new jobs have the lowest runtime)

---

## Problems with Basic CFS

- **Frequent context switches** add overhead (~1 us each)
- **Equal isn't always best:** need to *weight* processes differently
- **Long sleeping tasks:** what runtime value when waking up?

### Reducing Context Switches

Context switching frequently is inefficient.

Solution: *Maximum overrun, C*
- How much ahead of the lowest-time thread can you get, *C*?
- Run process Pa if `Pa.vruntime < Pb.vruntime - C`

Equivalent to a minimum *timeslice*.

### Fair Scheduling Example with C=5

| Job | Arrival(s) | Run time (s) | Completion time |
|-----|------------|---------------|-----------------|
| A   | 0          | 50            | 85              |
| B   | 10         | 15            | 35              |
| C   | 15         | 10            | 25              |
| B   | 50         | 10            | 60              |

> [!note] Lecture Note
> At time 15, runtimes are A: 10, B: 5, C: 5. At time 25, A: 10, B: 10. With C=5 the scheduler switches less often, improving turnaround time to 32.5 while response time remains 0.

**Average Turnaround Time:** (85-0) + (35-10) + (25-15) + (60-50) / 4 = **32.5**

**Average Response Time:** **0**

---

## Weighted Fair Scheduling

What if some processes are more important than others?

- Add *weight* to each process. More weight → more runtime
- Processes run in proportion to weights
  - A: 4, B: 2, C: 1 → runtime proportions 4:2:1

Calculate **virtual runtime** = real runtime / weight
- Run the process with the lowest virtual runtime

Overrun uses virtual runtime.

### Weighted Fair Scheduling Example (C=5)

| Job | Arrival(s) | Run time (s) | Weight |
|-----|------------|---------------|--------|
| A   | 0          | 50            | 4      |
| B   | 10         | 15            | 2      |
| C   | 15         | 10            | 1      |
| B   | 70         | 10            | 2      |

> [!note] Lecture Note
> A has weight 4, so its virtual runtime increases at 1/4 the rate of real time. B at 1/2 rate, C at full rate. A gets proportionally more real CPU time because its virtual runtime grows slower.

### Graphical View

Virtual runtime for a process advances when it runs:
- **Shallower** slope for processes with larger weight
- **Steeper** slope for processes with lower weight

The step between segments shows the maximum overrun.

> [!note] Lecture Note
> `vruntime = runtime / weight`. In the graph, gcc (higher weight) has a shallower line and bigsim (lower weight) has a steeper line. The weights of the two processes can be inferred from their slopes.

---

## Problem: Long-Sleeping Tasks

What happens when a sleeping task wakes?

| Job | Arrival(s) | Run time (s) | Weight |
|-----|------------|---------------|--------|
| A   | 0          | 5             | 1      |
| B   | 5          | 100           | 1      |
| A   | 50         | 50            | 1      |

A runs for 5s (vruntime = 5), then sleeps for 45s. Meanwhile B runs and accumulates vruntime = 45. When A wakes at time 50, its vruntime (5) is far below B's (45), so **A monopolizes the CPU** until it catches up.

> [!tip] Key Insight
> A sleeping task retains its old (low) vruntime. When it wakes, it unfairly dominates the CPU because the scheduler always picks the lowest vruntime process.

### Solution: Reset Vruntime

Calculate minimum vruntime for all **running** processes.

- If `P.vruntime > minimum` → do nothing (short sleep)
- If `P.vruntime < minimum` → set `P.vruntime = minimum - threshold D`
  - *D*: preemption threshold (>= C)

> [!example] Example
> With D = C: A wakes at time 50 with vruntime reset to `min(B.vruntime) - C = 45 - 5 = 40`. Now A and B alternate fairly: A runs 50-60 (vruntime 50), B runs 60-70 (vruntime 55), etc.

---

## Multiprocessor Scheduling

Thus far, assumed a single CPU core. With multiple CPUs, things get a lot more complicated.

### Symmetric Multiprocessing (SMP)

> **SMP:** 2 or more homogeneous processors (may be in separate physical packages), shared main memory and system bus, single OS that treats all processors equally.

> [!note] Lecture Note
> L3 cache is shared across cores within a CPU package. Each core has its own L1 and L2 caches.

### Hyperthreading

Two threads on a single CPU core.

- **Non-hyperthreaded core:** Thread 1 alternates between CPU Busy and Memory Stall — CPU is idle during stalls
- **Hyperthreaded core:** Thread 2 runs during Thread 1's memory stalls, filling idle gaps and improving throughput

### CPU Caches and Locality

Process performance is linked to **locality**:
- Ideally, a process should be placed close to its data

Shared data is problematic due to **cache coherency**:
- P3 writes variable x, new value is cached in CPU 2
- P2 in CPU 1 reads x, but value in main memory is stale

Cache hits are fast. A process has fast access to data on its own CPU, but access to another CPU's cached data is slow. Memory fetches are slow.

### NUMA and Affinity

> **NUMA (Non-Uniform Memory Access):** Architecture where memory access time depends on the location of the data relative to the requesting process.

**Cache affinity:** try to run processes close to their cached data.

---

## Single Queue Scheduling (SQMS)

> **SQMS (Single Queue Multiprocessor Scheduling):** Most basic design — all processes go into a single queue.

- CPUs pull tasks from the queue as needed
- Good for **load balancing** (CPUs pull processes on demand)

### Problems with SQMS

- The process queue is a shared data structure
  - Necessitates locking, or careful lock-free design
- SQMS does not respect **cache affinity**
  - Worst case: processes rarely run on the same CPU

---

## Multi-Queue Scheduling (MQMS)

> **MQMS (Multiple Queue Multiprocessor Scheduling):** Each CPU maintains its own queue of processes. CPUs schedule their processes independently.

### Advantages of MQMS

- Very little shared data — queues are (mostly) independent
- Respects **cache affinity** — processes stay on the same CPU

### Shortcomings of MQMS

MQMS is prone to **load imbalance** due to:
- Different number of processes per CPU
- Variable behavior across processes

Can lead to idle CPUs or unfair CPU usage.

### Process Migration

Must deal with load imbalance through **process migration**:

- **Push migration:** overloaded CPU sends a process to another CPU
  - "I have too many processes, take one"
- **Pull migration (work stealing):** underloaded CPU takes a process from another CPU's queue
  - "I don't have enough processes, give me one"

---

## Summary

- **Proportional share (fair) scheduling:** give processes equal CPU time
- **Pragmatics of fair scheduling:** overrun, weighting, reset for sleeping
- **Multiprocessor scheduling:** cache locality, affinity, and migration

---

## Related

- **Navigation:** [[lecture-03|← Lecture 3]] · [[lecture-05|Lecture 5 →]]
- **Exam:** [[exams/midterm-1/index|Midterm 1]]
- **Course:** [[README|CS 537 Map of Content]]