# Lecture 11 - Concurrency Intro (Threads)

Source: https://wisconsin.twango.dev/sp26-cs537/lectures/lecture-11

# Lecture 11 - Concurrency Intro (Threads)

**Instructor:** Tej Chajed
**Topics:** Concurrency, threads, race conditions, synchronization, locks

> [!abstract]- Learning Outcomes
> - Articulate the difference between processes and threads
> - Explain why concurrency is hard
> - Appreciate why we want things to be atomic

---

## Motivation for Concurrency

CPU performance growth has slowed dramatically:
- 1978-1986: **25%/year** improvement
- 1986-2003: **52%/year** improvement
- 2003-2011: **23%/year**
- 2011-2015: **12%/year**
- 2015-2018: **3.5%/year**

Moore's Law continues — transistor counts still double roughly every two years — but individual core speed has plateaued.

CPU trend: **Same speed, but multiple cores** (64 cores as of 2019)

Goal: Write applications that fully utilize many cores

---

## Approaches to Concurrency

### Option 1: Multiple Processes

Build apps from many communicating **processes**
- Example: Chrome (process per tab)
- Communicate via `pipe()` or similar

Pros:
- Don't need new abstractions; good for security

Cons:
- Cumbersome programming
- High communication overheads
- Expensive context switching (requires changing address space)

### Option 2: Threads

New abstraction: **thread**

Threads are like processes, except: **multiple threads of the same process share an address space**

- Divide large task across several cooperative threads
- Communicate through shared address space

---

## Thread vs. Process

### Shared State

Multiple threads within a single process share:
- Process ID (PID)
- Address space: code (instructions), most data (heap)
- Open file descriptors
- Current working directory
- User and group id

### Per-Thread State

Each thread has its own:
- Thread ID (TID)
- Set of registers, including program counter and stack pointer
- Stack for local variables and return addresses (in same address space)

> [!note] Lecture Note
> Threads share the same PTBR (page table base register) — they point to the same page directory. But each thread has its own IP (instruction pointer) since they may be executing different code at the same time. Each thread also has its own SP (stack pointer) since threads executing different functions need different stacks.

### Process Address Space with Threads

Single-threaded process address space:

| Region | Description |
|--------|-------------|
| Code (text segment) | Program instructions (PC points here) |
| Static data (data segment) | Global variables |
| Heap | Dynamically allocated memory (grows down) |
| Stack | Local variables and return addresses (grows up) |

With multiple threads, the address space now contains **multiple stacks** — one per thread. Each thread has its own SP pointing to its own stack, and its own PC pointing into the shared code segment.

---

## Thread Context Switch

Like process context switch:
- Trap to kernel
- Save context of currently running thread
  - Push machine state onto thread stack
- Restore context of the next thread
  - Pop machine state from next thread's stack
- Return as the new thread
  - Execution resumes at PC of next thread

What's **not** done:
- Change address space (no need — threads share the same one)

---

## Thread API

Variety of thread systems exist:
- POSIX Pthreads

Common thread operations:
- **Create** — start a new thread
- **Exit** — terminate current thread
- **Join** — wait for a thread to finish (instead of `wait()` for processes)

Notice: no `fork()` — why not? Threads share the address space, so there's no need to duplicate it.

---

## OS Support for Threads

### User-Level Threads (Many-to-One)

- Implemented by user-level runtime libraries
- Create, schedule, synchronize threads at user-level
- OS is not aware of user-level threads — OS thinks each process contains only a single thread of control

Advantages:
- Does not require OS support; portable
- Lower overhead thread operations since no system call

Disadvantages:
- Cannot leverage multiprocessors
- Entire process blocks when one thread blocks

### Kernel-Level Threads (One-to-One)

- OS provides each user-level thread with a kernel thread
- Each kernel thread scheduled independently
- Thread operations (creation, scheduling, synchronization) performed by OS

Advantages:
- Each kernel-level thread can run in parallel on a multiprocessor
- When one thread blocks, other threads from process can be scheduled

Disadvantages:
- Higher overhead for thread operations
- OS must scale well with increasing number of threads

---

## Race Conditions

### The Problem: `balance++`

```c
volatile int balance = 0;
int loops;

void *worker(void *arg) {
    int i;
    for (i = 0; i < loops; i++) {
        balance++;
    }
    pthread_exit(NULL);
}

int main(int argc, char *argv[]) {
    loops = atoi(argv[1]);
    pthread_t p1, p2;
    printf("Initial value : %d\n", balance);
    Pthread_create(&p1, NULL, worker, NULL);
    Pthread_create(&p2, NULL, worker, NULL);
    Pthread_join(p1, NULL);
    Pthread_join(p2, NULL);
    printf("Final value   : %d\n", balance);
    return 0;
}
```

> [!note] Lecture Note
> Running `./threads 100000` gives Initial value: 0, Final value: 162901 — not the expected 200000!

The statement `balance++` compiles to three assembly instructions:

```text
0x195  mov 0x9cd4, %eax    // load balance into register
0x19a  add $0x1, %eax      // increment register
0x19d  mov %eax, 0x9cd4    // store register back to balance
```

### Schedule #1 (Correct Result)

Thread 1 runs all three instructions, then context switches to Thread 2:

| Step | Thread | Instruction | %eax | balance (0x9cd4) |
|------|--------|-------------|------|------------------|
| 1 | T1 | `mov 0x9cd4, %eax` | 100 | 100 |
| 2 | T1 | `add $0x1, %eax` | 101 | 100 |
| 3 | T1 | `mov %eax, 0x9cd4` | 101 | **101** |
| — | — | *context switch* | — | — |
| 4 | T2 | `mov 0x9cd4, %eax` | 101 | 101 |
| 5 | T2 | `add $0x1, %eax` | 102 | 101 |
| 6 | T2 | `mov %eax, 0x9cd4` | 102 | **102** |

Final balance: **102** (correct — incremented twice)

### Schedule #2 (Wrong Result)

Thread 1 executes load and add, then context switches to Thread 2 **before the store**:

| Step | Thread | Instruction | %eax | balance (0x9cd4) |
|------|--------|-------------|------|------------------|
| 1 | T1 | `mov 0x9cd4, %eax` | 100 | 100 |
| 2 | T1 | `add $0x1, %eax` | 101 | 100 |
| — | — | *context switch* | — | — |
| 3 | T2 | `mov 0x9cd4, %eax` | 100 | 100 |
| 4 | T2 | `add $0x1, %eax` | 101 | 100 |
| 5 | T2 | `mov %eax, 0x9cd4` | 101 | **101** |
| — | — | *context switch* | — | — |
| 6 | T1 | `mov %eax, 0x9cd4` | 101 | **101** |

Final balance: **101** (wrong — one increment lost)

> [!tip] Key Insight
> T1 loaded the old value (100) into its register, then got interrupted. T2 read the same old value and wrote back 101. When T1 resumes, it writes its stale value (101), overwriting T2's update.

---

## Timeline View

The interleaving of instructions determines correctness. Thread 1 adds 1, Thread 2 adds 2 to a shared variable:

**Correct interleaving** — T1 completes, then T2 runs:

```text
Thread 1              Thread 2
mov 0x123, %eax
add %0x1, %eax
mov %eax, 0x123
                      mov 0x123, %eax
                      add %0x2, %eax
                      mov %eax, 0x123
```

Result: **3 added** (correct)

**Incorrect interleaving** — T2 reads before T1 writes back:

```text
Thread 1              Thread 2
mov 0x123, %eax
add %0x1, %eax
                      mov 0x123, %eax
mov %eax, 0x123
                      add %0x2, %eax
                      mov %eax, 0x123
```

Result: **2 added** (incorrect — T1's update lost)

**Another incorrect interleaving:**

```text
Thread 1              Thread 2
                      mov 0x123, %eax
mov 0x123, %eax
                      add %0x2, %eax
add %0x1, %eax
                      mov %eax, 0x123
mov %eax, 0x123
```

Result: **1 added** (incorrect — T2's update lost)

---

## Scheduling: Possible Outcomes (Linked List)

```c
struct node {
    struct node* next;
    int data;
}
struct node* head;
```

Thread 1 inserts node2, Thread 2 inserts node3 at the head of the list:

```c
// Thread 1:              // Thread 2:
node2->next = head;        node3->next = head;
head = node2;              head = node3;
```

Initially: `head = node1`. Possible outcomes after both threads run:

1. **Head → Node 3 → Node 2 → Node 1** (T1 runs first, then T2 — correct)
2. **Head → Node 2 → Node 3 → Node 1** (T2 runs first, then T1 — different order, but correct)
3. **Missing/leaked nodes** — if both threads read `head` before either writes, one node's insertion is lost (node2 or node3 becomes unreachable)

---

## How Race Conditions Happen

### Multiple CPUs

Two threads run simultaneously on different CPUs, both accessing the same memory location at the same time.

### Interrupts

On a single CPU, a timer interrupt causes a context switch in the middle of a critical section. Thread 1 is interrupted between its load and store, and Thread 2 runs its entire critical section before T1 resumes.

---

## Non-Determinism

Concurrency leads to non-deterministic results:
- Different results even with same inputs
- **Race conditions**

Whether a bug manifests depends on the CPU schedule!

How to program: imagine the scheduler is malicious?!

---

## What Do We Want?

Want the 3 instructions (load, add, store) to *appear to execute* as an uninterruptible group — that is, we want them to be **atomic**.

```text
mov 0x123, %eax
add %0x1, %eax
mov %eax, 0x123
```

> **Critical section:** A piece of code that accesses a shared resource and must not be concurrently executed by more than one thread.

More general: Need **mutual exclusion** for critical sections — if thread A is in critical section C, thread B isn't (okay if other threads do unrelated work).

---

## Synchronization

Build higher-level synchronization primitives in OS. Operations that ensure correct ordering of instructions across threads. Use help from hardware.

Motivation: Build them once and get them right.

| Layer | Primitives |
|-------|-----------|
| High-level | Monitors, Locks, Condition Variables, Semaphores |
| Low-level (hardware) | Loads, Stores, Disable Interrupts, Test&Set |

---

## Locks

Goal: Provide **mutual exclusion** (mutex)

### Lock API

Allocate and initialize:
- `Pthread_mutex_t mylock = PTHREAD_MUTEX_INITIALIZER;`

Acquire:
- Acquire exclusive access to lock
- Wait if lock is not available (some other process in critical section)
- Spin or block (relinquish CPU) while waiting
- `Pthread_mutex_lock(&mylock);`

Release:
- Release exclusive access to lock; let another process enter critical section
- `Pthread_mutex_unlock(&mylock);`

### Timeline View with Locks

```text
Thread 1              Thread 2
mutex_Lock()          mutex_Lock()
mov 0x123, %eax
add %0x1, %eax
mov %eax, 0x123
mutex_unlock()
                      mov 0x123, %eax
                      add %0x2, %eax
                      mov %eax, 0x123
                      mutex_unlock()
```

Result: **3 added** (always correct — lock prevents interleaving)

### Lock Implementation Goals

Correctness:
- **Mutual exclusion** — only one thread in critical section at a time
- **Progress** (deadlock-free) — if several simultaneous requests, must allow one to proceed
- **Bounded** (starvation-free) — must eventually allow each waiting thread to enter

Fairness: Each thread waits for same amount of time

Performance: CPU is not used unnecessarily

---

## Implementing Locks

> **Atomic operation:** No other instructions can be interleaved.

Approaches:
1. Disable interrupts
2. Locks using loads/stores
3. Using special hardware instructions

### Approach 1: Disable Interrupts

Turn off interrupts for critical sections:
- Prevent dispatcher from running another thread
- Code between interrupts executes atomically

```c
void acquire(lockT *l) {
    disableInterrupts();
}

void release(lockT *l) {
    enableInterrupts();
}
```

Disadvantages:
- Only works on uniprocessors
- Process can keep control of CPU for arbitrary length
- Cannot perform other necessary work

### Approach 2: Load+Store

Code uses a single **shared** lock variable:

```c
// shared variable
boolean lock = false;
void acquire(Boolean *lock) {
    while (*lock) /* wait */ ;
    *lock = true;
}

void release(Boolean *lock) {
    *lock = false;
}
```

**Does this work?** No — race condition with load and store:

```text
*lock == 0 initially

Thread 1              Thread 2
while(*lock == 1)
                      while(*lock == 1)
                      *lock = 1
*lock = 1
```

**Both threads grab the lock!** Problem: testing and setting are not atomic.

### Atomic Operations

To implement synchronization, we need atomic operations:
- Code between interrupts on uniprocessors
  - Disable timer interrupts, don't do any I/O
- Loads and stores of single words
  - `Load r1, B`
  - `Store r1, A`
- **Special hardware instructions**
  - Test&Set
  - Compare&Swap

---

## Related

- **Navigation:** [[lecture-10|← Lecture 10]] · [[lecture-12|Lecture 12 →]]
- **Exam:** [[exams/midterm-2/index|Midterm 2]]
- **Course:** [[README|CS 537 Map of Content]]