# Lecture 16 - Persistence: I/O Devices

Source: https://wisconsin.twango.dev/sp26-cs537/lectures/lecture-16

# Lecture 16 - Persistence: I/O Devices

**Instructor:** Michael Swift
**Topics:** I/O devices, device drivers, polling vs. interrupts, DMA, hard disks, solid-state drives, flash translation layer

---

## Operating Systems: Three Easy Pieces

Three conceptual pieces of an OS:

1. **Virtualization** — make each application believe it has each resource to itself (CPU and memory)
2. **Concurrency** — provide mutual exclusion and ordering
3. **Persistence** — store data reliably across power cycles

---

## Motivation for I/O Devices

What good is a computer without any I/O devices? (keyboard, display, disks)

We want:
- **Hardware** that will let us plug in different devices
- **OS** that can interact with different combinations of devices

---

## Hardware Support for I/O

A system has a hierarchy of buses connecting devices:

```mermaid
graph TD
    CPU[CPU] --- MB[Memory Bus]
    RAM[RAM] --- MB
    MB --- GIO[General I/O Bus - e.g., PCIe]
    GIO --- Graphics[Graphics]
    GIO --- PIO[Peripheral I/O Bus - e.g., SCSI, SATA, USB]
    PIO --- D1[Disk]
    PIO --- D2[Disk]
    PIO --- D3[Disk]
    PIO --- D4[Disk]
```

> [!note] Lecture Note
> The memory bus is the fastest. Devices further down the hierarchy are on slower buses.

---

## Canonical Device

Every device presents two parts to the OS:

| Component | Description |
|-----------|-------------|
| **Device Registers** | Status, Command, and Data registers that the OS reads/writes |
| **Hidden Internals** | Microcontroller (CPU+RAM), extra RAM, other special-purpose chips |

The OS interacts with the device by reading and writing its registers. The internals are hidden from the OS.

> [!note] Lecture Note
> Device registers are mapped to physical addresses. DRAM occupies part of the physical address space (accessed via memory bus), and I/O devices occupy another part (accessed via I/O bus).

---

## Example Write Protocol (Polling)

The basic protocol for writing to a device using **polling**:

```text
while (STATUS == BUSY)    // 1 - poll until device is free
    ;                     // spin
Write data to DATA register       // 2
Write command to COMMAND register // 3
while (STATUS == BUSY)    // 4 - poll until device completes
    ;                     // spin
```

> [!note] Lecture Note
> Steps 1-2 form a loop when there is data to transfer. The CPU is stuck spinning while waiting for the device, wasting cycles.

> [!example] Example
> Process A wants to do I/O while the disk is busy with C. A spins at step 1 until C finishes, then writes data (step 2), issues the command (step 3), and spins again at step 4 waiting for the device to finish. During all that spinning, no other process gets to run — the CPU is wasted.

---

## Interrupts: Avoiding Spinning

Instead of polling (spinning), the device can send an **interrupt** when it is done:

```text
while (STATUS == BUSY)            // 1
    wait for interrupt;
Write data to DATA register       // 2
Write command to COMMAND register // 3
while (STATUS == BUSY)            // 4
    wait for interrupt;
```

With interrupts, when process A issues I/O and the device is busy, the OS can switch to process B. When the device finishes, it raises an interrupt, the **interrupt handler** runs, and A is woken up.

> [!note] Lecture Note
> The interrupt handler calls `wakeup()` to move the waiting process back to the ready queue. This lets the CPU do useful work (run B) while the device is busy.

---

## Example Devices

| Device | Requests | Data |
|--------|----------|------|
| **Storage** (SSDs) | Reading/writing location on drive | Block in memory |
| **Network** (wifi, ethernet) | Send, receive packet | Packet in memory |
| **Keyboard**, mouse | Reading keystroke, mouse movement | Key pressed, movement direction/distance |

---

## Event Types

**Synchronous:**
- Launched by OS, completed by device
  - Read/write data to storage
  - Send a network packet
  - Move a robot arm

**Asynchronous:**
- Triggered externally, request service from OS
  - Receive a packet
  - Keystroke
  - Mouse movement

Polling works well for synchronous events. For asynchronous events (like keystrokes), interrupts are essential — the OS registers an interrupt handler that reads the device registers when the event occurs.

```text
keyboard_interrupt_handler() {
    read STATUS register  // key up or down
    read DATA register    // which key
}
```

---

## Interrupts vs. Polling

Are interrupts always better than polling?

- **Fast device:** Better to spin than take interrupt overhead
  - Device time unknown? Use a **hybrid approach** (spin first, then use interrupts)
- **Flood of interrupts:** Can lead to **livelock** (always handling interrupts, never making progress)
  - Better to ignore interrupts while making some progress handling them
- **Interrupt coalescing:** Batch together several interrupts into one

---

## Protocol Variants

Three axes of variation for device protocols:

| Variant | Options |
|---------|---------|
| **Status checks** | Polling vs. interrupts |
| **Data transfer** | PIO vs. DMA |
| **Batching** | 1 request at a time or a queue |

---

## Programmed I/O vs. Direct Memory Access

**PIO** (Programmed I/O):
- CPU directly tells device what the data is
- CPU writes data to the DATA register (step 2 in the protocol)
- CPU is busy during the data transfer

**DMA** (Direct Memory Access):
- CPU leaves data in memory
- Device reads data directly from memory
- CPU writes a **memory address** to the DATA register instead of the actual data
- Frees the CPU — device reads data on its own, then does I/O

> [!note] Lecture Note
> With DMA, step 2 becomes "write address to DATA register" instead of "write data to DATA register." The device reads the data from memory, allowing the CPU to do other work. Steps 2, 3, and 4 become very fast since the CPU only writes an address and a command.

---

## Submitting Multiple Requests

Challenge:
- Multiple requests with different data locations (e.g., multiple network packets)
- Multiple status outcomes (some failure, some success)

Solution: **Command and Completion queues** with descriptors

### I/O Request Queues (Batching)

- I/O commands are added to a circular **submission queue** (in memory)
- Driver writes to the **SQ doorbell** register when adding an entry, giving the index
- Device processes commands and writes status to a **completion queue**
- Device puts latest entry index in the **CQ doorbell**

> [!note] Lecture Note
> The driver and device communicate through shared memory queues rather than individual register writes. This is the approach used by NVMe SSDs.

---

## Device Drivers

Problem: many devices, each with its own protocol. How can we avoid writing a slightly different OS for each H/W combination?

Solution: write a **device driver** for each device.

- Drivers are **70%** of Linux source code
- Each OS defines a *class interface* for common device types (network, block, sound)
- Driver exports the class interface to the OS
- Driver uses a device-specific interface to talk to the device
- Some devices share common device interfaces (e.g., USB storage, SATA, NVMe for storage; keyboard/mouse)
- Must be written separately for each OS (Linux, Windows, macOS)

```mermaid
graph TD
    K[OS Kernel] --- KI[Kernel Interface]
    KI --- CI[Driver Class Interface]
    CI --- DD[Device-Specific Driver]
    DD --- HW[Hardware Device]
```

---

## Hard Disk Interface

A hard disk has a **sector-addressable** address space:
- Appears as an **array of sectors** (e.g., `sector[43]`)
- Adjacent sectors have adjacent numbers
- Sectors are typically **4096 bytes**
- Main operations: reads + writes to sectors (e.g., "read 20 sectors starting at sector 242")
- Mechanical and slow

> [!note] Lecture Note
> The disk interface is simple — just an array of sectors. But physical access time depends heavily on where the data is located.

---

## Hard Disk Components

- **Platter** — circular disk covered with magnetic film; stores data
- Multiple platters can be stacked
- **Read/write head** — on a moving **arm**, can read from each surface
- Data is organized in concentric **tracks** on each platter surface
- Each track is divided into sectors

---

## Reading Data from Disk

Two types of delay to access a sector:

1. **Seek time** — time for the arm to move the head to the correct track
2. **Rotational delay** — time for the platter to spin until the desired sector is under the head

### Time to Read/Write

Three components:

**Time = seek + rotation + transfer time**

Typical values:

| Component | Typical Value |
|-----------|---------------|
| Seek | 4-9 ms |
| Rotation | 4-8 ms (depends on RPM: 5400, 7200, 10000, or 15000 RPM) |
| Transfer | 5 ms/MB or 0.005 ms/KB |

> [!note] Lecture Note
> Smaller disk diameter means shorter seek times. Rotation speed in RPM determines rotational latency — e.g., at 3000 RPM (50 RPS), worst-case rotational delay is 1/50s = 20ms.

---

## Disk Performance Implications

If the disk head is in the **right place**:
- Read next 1 KB takes **0.005 ms**

If the disk head is in the **wrong place**:
- Read next 1 KB takes **8-17 ms** — **3400x slower**

How do we keep the disk head in the right place?
- Access blocks **sequentially** on a track — no rotation delay, no seek

> [!tip] Key Insight
> Sequential access on a hard disk is orders of magnitude faster than random access. This is why file systems try to allocate files contiguously on disk.

---

## Solid-State Storage (SSDs)

Hard drives use mechanical mechanisms to find data — inherently slow for modern GHz processors.

**Solid-state drives** use electronics, chips fabricated similarly to CPUs. Speeds are closer to CPU speeds (but not all the way).

Unlike hard drives, SSDs have **no mechanical parts**:
- SSDs use transistors (just like DRAM), but SSD data **persists when the power goes out**
- **NAND-based flash** is the most popular technology

High-level takeaways:
1. SSDs have a higher $/bit than hard drives, but **better performance** (no mechanical delays)
2. SSDs handle writes in a strange way; this has implications for file system design

> [!note] Lecture Note
> The chips on an SSD are flash memory chips.

---

## NAND Flash Cell Types

A NAND flash block is a grid of cells. Levels are distinct voltage levels in a single cell.

| Cell Type | Bits per Cell | Characteristics |
|-----------|--------------|-----------------|
| **SLC** (Single Level Cell) | 1 | Faster, more reliable |
| **MLC** (Multi Level Cell) | 2 | Slower, less reliable |
| **TLC** (Triple Level Cell) | 3 | Even more so |
| **QLC** (Quad Level Cell) | 4 | Slowest, least reliable |

Basic flash operations:
- **Erase:** set all bits to 1
- **Program (write):** clear selected bits to 0
- **Read:** read value of bits

---

## SSD Structure

An SSD contains **blocks** made of **pages**:
- A **page** is a few KB in size (e.g., 4 KB)
- A **block** contains several pages, is usually 128 KB or 256 KB

Key constraints:
- To write a single page, you must **erase the entire block first**
- A block is likely to **fail after a certain number of erases** (~1000 for highest-density flash, ~100,000 for lowest-density flash)

### SSD Internal Architecture

The SSD contains:
- **Host interface** (SATA, SCSI, NVMe, etc.)
- **DRAM** for caching and mapping tables
- **Embedded processor** running the Flash Translation Layer (FTL) firmware
- **Low-level flash controller** connected to multiple NAND flash chips via flash bus

### Striping

- Page addresses (like sectors) are like array indexes into the SSD
- Page addresses are **striped across flash packages**
- A single request can span multiple chips — natural load balancing

---

## SSD Operations (Latency)

| Operation | Latency | Notes |
|-----------|---------|-------|
| **Read** a page | 25-75 microseconds | Cost is independent of page number or prior requests |
| **Erase** a block | 1.5-4.5 milliseconds | Much more expensive than reading; resets all pages to 1s |
| **Program** (write) a page | 200-1400 microseconds | Faster than erasing, but slower than reading |

For comparison — hard disk: 4-9 ms avg. seek latency, 4-7 ms avg. rotational latency.

---

## The Write Problem

To write a single page on an SSD:
1. The entire block must first be **erased** (all bits set to 1)
2. Then the page can be **programmed** (selected bits set to 0)
3. But erasing destroys all other pages in the block

This is handled by the **Flash Translation Layer (FTL)**.

---

## Flash Translation Layer (FTL)

The FTL is firmware that manages the SSD. It has three goals:

1. **Translate** reads/writes to logical blocks into reads/erases/programs on physical pages+blocks
   - Allows SSDs to export the simple "block interface" that hard disks have traditionally exported
   - Hides write-induced copying and garbage collection from applications
2. **Reduce write amplification** (the amount of extra copying needed to deal with block-level erases)
3. **Implement wear leveling** (distribute writes equally to all blocks, to avoid fast failures of a "hot" block)

FTL is typically implemented in hardware in the SSD, but is implemented in software for some SSDs.

### FTL Approach #1: Direct Mapping

Have a 1-1 correspondence between logical pages and physical pages.

- **Reading** a page is straightforward
- **Writing** a page is expensive:
  1. Read the entire physical block into memory
  2. Update the relevant page in the in-memory block
  3. Erase the entire physical block
  4. Program the entire physical block using the new block value

**Problems with direct mapping:**
- **Write amplification** — writing a single page requires reading and writing an entire block (slow)
- **Poor reliability** — if the same logical block is repeatedly written, its physical block will quickly fail
- **Data loss risk** — if failure happens between erase and rewrite, data is lost

### FTL Approach #2: Log-Based Mapping

Basic idea: treat the physical blocks like a **log**.

- Write data in each page-to-write to the **end of the log**
- Maintain a **logical-to-physical mapping** between logical pages and their corresponding physical pages in the SSD

How a write works:
1. If the log head is at the start of a new block, erase that block
2. Program the page at the log head with the new data
3. Update the logical-to-physical map
4. Advance the log head

> [!example] Example
> Writing logical page 92 with data w0: erase block 0, program physical page 0 with w0, map 92 --> 0, advance log head. Writing logical page 17 with data w1: program physical page 1 with w1, map 17 --> 1, advance log head. No erase needed since we are still in the same block.

**Advantages over direct mapping:**
- Avoids expensive read-modify-write behavior
- Better **wear leveling**: writes get spread across pages, even if there is spatial locality in writes at the logical level

---

## Garbage Collection

When a logical page is overwritten, the old physical page becomes **garbage** (stale data). Over time, garbage pages accumulate.

Challenge:
- Overwritten pages take up space
- May share a block with valid data

Solution: **garbage collection**
1. Read valid data from a block (ignore garbage)
2. Write valid data to end of log
3. Update the logical-to-physical map
4. Erase the old block — it is now free space

Garbage collection requires extra read+write traffic.

### Overprovisioning

**Overprovisioning** makes GC less painful:
- SSD exposes a logical page space that is **smaller** than the physical page space
- By keeping extra "hidden" pages around, the SSD tries to defer GC to a background task (removing GC from the critical path of a write)

The SSD will occasionally shuffle live (non-garbage) blocks that never get overwritten — this enforces **wear leveling**.

---

## SSDs vs. Hard Drives (Throughput)

| Device | Random Reads (MB/s) | Random Writes (MB/s) | Sequential Reads (MB/s) | Sequential Writes (MB/s) |
|--------|---------------------|----------------------|------------------------|-------------------------|
| Samsung 960 Evo Plus SSD | 85.5 | 244 | 2664 | 2508 |
| Crucial BX100 SSD | 24.5 | 73.5 | 466 | 392 |
| Samsung 840 EVO SSD | 44.1 | 112 | 502 | 494 |
| Seagate Savio 15K.3 HD | 2 | 2 | 223 | 223 |

> [!note] Lecture Note
> SSDs dominate in random reads/writes and sequential performance. But hard drives are ~10x cheaper per storage bit. Note how the hard drive has essentially the same throughput for random and sequential access — it is bottlenecked by mechanical seek/rotation.

---

## SSD Summary

- Most local storage will be solid state — faster, cheap enough
- Best for **fast random access**
- Relies on FTL for fast writes
  - Sequential and random writes have closer cost (unlike hard disks)
  - Garbage collection is more expensive with random writes

---

## Related

- **Navigation:** [[lecture-15|← Lecture 15]]
- **Exam:** [[exams/midterm-2/index|Midterm 2]]
- **Course:** [[README|CS 537 Map of Content]]