Lecture 16 - Persistence: I/O Devices

Instructor: Michael Swift Topics: I/O devices, device drivers, polling vs. interrupts, DMA, hard disks, solid-state drives, flash translation layer

Operating Systems: Three Easy Pieces

Three conceptual pieces of an OS:

1. Virtualization — make each application believe it has each resource to itself (CPU and memory)
2. Concurrency — provide mutual exclusion and ordering
3. Persistence — store data reliably across power cycles

Motivation for I/O Devices

What good is a computer without any I/O devices? (keyboard, display, disks)

We want:

• Hardware that will let us plug in different devices
• OS that can interact with different combinations of devices

Hardware Support for I/O

A system has a hierarchy of buses connecting devices:

graph TD
    CPU[CPU] --- MB[Memory Bus]
    RAM[RAM] --- MB
    MB --- GIO[General I/O Bus - e.g., PCIe]
    GIO --- Graphics[Graphics]
    GIO --- PIO[Peripheral I/O Bus - e.g., SCSI, SATA, USB]
    PIO --- D1[Disk]
    PIO --- D2[Disk]
    PIO --- D3[Disk]
    PIO --- D4[Disk]

Lecture Note

The memory bus is the fastest. Devices further down the hierarchy are on slower buses.

Canonical Device

Every device presents two parts to the OS:

The OS interacts with the device by reading and writing its registers. The internals are hidden from the OS.

Lecture Note

Device registers are mapped to physical addresses. DRAM occupies part of the physical address space (accessed via memory bus), and I/O devices occupy another part (accessed via I/O bus).

Example Write Protocol (Polling)

The basic protocol for writing to a device using polling:

while (STATUS == BUSY)    // 1 - poll until device is free
    ;                     // spin
Write data to DATA register       // 2
Write command to COMMAND register // 3
while (STATUS == BUSY)    // 4 - poll until device completes
    ;                     // spin

Lecture Note

Steps 1-2 form a loop when there is data to transfer. The CPU is stuck spinning while waiting for the device, wasting cycles.

Example

Process A wants to do I/O while the disk is busy with C. A spins at step 1 until C finishes, then writes data (step 2), issues the command (step 3), and spins again at step 4 waiting for the device to finish. During all that spinning, no other process gets to run — the CPU is wasted.

Interrupts: Avoiding Spinning

Instead of polling (spinning), the device can send an interrupt when it is done:

while (STATUS == BUSY)            // 1
    wait for interrupt;
Write data to DATA register       // 2
Write command to COMMAND register // 3
while (STATUS == BUSY)            // 4
    wait for interrupt;

With interrupts, when process A issues I/O and the device is busy, the OS can switch to process B. When the device finishes, it raises an interrupt, the interrupt handler runs, and A is woken up.

Lecture Note

The interrupt handler calls wakeup() to move the waiting process back to the ready queue. This lets the CPU do useful work (run B) while the device is busy.

Example Devices

Event Types

Synchronous:

• Launched by OS, completed by device
  • Read/write data to storage
  • Send a network packet
  • Move a robot arm

Asynchronous:

• Triggered externally, request service from OS
  • Receive a packet
  • Keystroke
  • Mouse movement

Polling works well for synchronous events. For asynchronous events (like keystrokes), interrupts are essential — the OS registers an interrupt handler that reads the device registers when the event occurs.

keyboard_interrupt_handler() {
    read STATUS register  // key up or down
    read DATA register    // which key
}

Interrupts vs. Polling

Are interrupts always better than polling?

• Fast device: Better to spin than take interrupt overhead
  • Device time unknown? Use a hybrid approach (spin first, then use interrupts)
• Flood of interrupts: Can lead to livelock (always handling interrupts, never making progress)
  • Better to ignore interrupts while making some progress handling them
• Interrupt coalescing: Batch together several interrupts into one

Protocol Variants

Three axes of variation for device protocols:

Programmed I/O vs. Direct Memory Access

PIO (Programmed I/O):

• CPU directly tells device what the data is
• CPU writes data to the DATA register (step 2 in the protocol)
• CPU is busy during the data transfer

DMA (Direct Memory Access):

• CPU leaves data in memory
• Device reads data directly from memory
• CPU writes a memory address to the DATA register instead of the actual data
• Frees the CPU — device reads data on its own, then does I/O

Lecture Note

With DMA, step 2 becomes “write address to DATA register” instead of “write data to DATA register.” The device reads the data from memory, allowing the CPU to do other work. Steps 2, 3, and 4 become very fast since the CPU only writes an address and a command.

Submitting Multiple Requests

Challenge:

• Multiple requests with different data locations (e.g., multiple network packets)
• Multiple status outcomes (some failure, some success)

Solution: Command and Completion queues with descriptors

I/O Request Queues (Batching)

• I/O commands are added to a circular submission queue (in memory)
• Driver writes to the SQ doorbell register when adding an entry, giving the index
• Device processes commands and writes status to a completion queue
• Device puts latest entry index in the CQ doorbell

Lecture Note

The driver and device communicate through shared memory queues rather than individual register writes. This is the approach used by NVMe SSDs.

Device Drivers

Problem: many devices, each with its own protocol. How can we avoid writing a slightly different OS for each H/W combination?

Solution: write a device driver for each device.

• Drivers are 70% of Linux source code
• Each OS defines a class interface for common device types (network, block, sound)
• Driver exports the class interface to the OS
• Driver uses a device-specific interface to talk to the device
• Some devices share common device interfaces (e.g., USB storage, SATA, NVMe for storage; keyboard/mouse)
• Must be written separately for each OS (Linux, Windows, macOS)

graph TD
    K[OS Kernel] --- KI[Kernel Interface]
    KI --- CI[Driver Class Interface]
    CI --- DD[Device-Specific Driver]
    DD --- HW[Hardware Device]

Hard Disk Interface

A hard disk has a sector-addressable address space:

• Appears as an array of sectors (e.g., sector[43])
• Adjacent sectors have adjacent numbers
• Sectors are typically 4096 bytes
• Main operations: reads + writes to sectors (e.g., “read 20 sectors starting at sector 242”)
• Mechanical and slow

Lecture Note

The disk interface is simple — just an array of sectors. But physical access time depends heavily on where the data is located.

Hard Disk Components

• Platter — circular disk covered with magnetic film; stores data
• Multiple platters can be stacked
• Read/write head — on a moving arm, can read from each surface
• Data is organized in concentric tracks on each platter surface
• Each track is divided into sectors

Reading Data from Disk

Two types of delay to access a sector:

1. Seek time — time for the arm to move the head to the correct track
2. Rotational delay — time for the platter to spin until the desired sector is under the head

Time to Read/Write

Three components:

Time = seek + rotation + transfer time

Typical values:

Lecture Note

Smaller disk diameter means shorter seek times. Rotation speed in RPM determines rotational latency — e.g., at 3000 RPM (50 RPS), worst-case rotational delay is 1/50s = 20ms.

Disk Performance Implications

If the disk head is in the right place:

• Read next 1 KB takes 0.005 ms

If the disk head is in the wrong place:

• Read next 1 KB takes 8-17 ms — 3400x slower

How do we keep the disk head in the right place?

• Access blocks sequentially on a track — no rotation delay, no seek

Key Insight

Sequential access on a hard disk is orders of magnitude faster than random access. This is why file systems try to allocate files contiguously on disk.

Solid-State Storage (SSDs)

Hard drives use mechanical mechanisms to find data — inherently slow for modern GHz processors.

Solid-state drives use electronics, chips fabricated similarly to CPUs. Speeds are closer to CPU speeds (but not all the way).

Unlike hard drives, SSDs have no mechanical parts:

• SSDs use transistors (just like DRAM), but SSD data persists when the power goes out
• NAND-based flash is the most popular technology

High-level takeaways:

1. SSDs have a higher $/bit than hard drives, but better performance (no mechanical delays)
2. SSDs handle writes in a strange way; this has implications for file system design

Lecture Note

The chips on an SSD are flash memory chips.

NAND Flash Cell Types

A NAND flash block is a grid of cells. Levels are distinct voltage levels in a single cell.

Basic flash operations:

• Erase: set all bits to 1
• Program (write): clear selected bits to 0
• Read: read value of bits

SSD Structure

An SSD contains blocks made of pages:

• A page is a few KB in size (e.g., 4 KB)
• A block contains several pages, is usually 128 KB or 256 KB

Key constraints:

• To write a single page, you must erase the entire block first
• A block is likely to fail after a certain number of erases (~1000 for highest-density flash, ~100,000 for lowest-density flash)

SSD Internal Architecture

The SSD contains:

• Host interface (SATA, SCSI, NVMe, etc.)
• DRAM for caching and mapping tables
• Embedded processor running the Flash Translation Layer (FTL) firmware
• Low-level flash controller connected to multiple NAND flash chips via flash bus

Striping

• Page addresses (like sectors) are like array indexes into the SSD
• Page addresses are striped across flash packages
• A single request can span multiple chips — natural load balancing

SSD Operations (Latency)

For comparison — hard disk: 4-9 ms avg. seek latency, 4-7 ms avg. rotational latency.

The Write Problem

To write a single page on an SSD:

1. The entire block must first be erased (all bits set to 1)
2. Then the page can be programmed (selected bits set to 0)
3. But erasing destroys all other pages in the block

This is handled by the Flash Translation Layer (FTL).

Flash Translation Layer (FTL)

The FTL is firmware that manages the SSD. It has three goals:

1. Translate reads/writes to logical blocks into reads/erases/programs on physical pages+blocks
   • Allows SSDs to export the simple “block interface” that hard disks have traditionally exported
   • Hides write-induced copying and garbage collection from applications
2. Reduce write amplification (the amount of extra copying needed to deal with block-level erases)
3. Implement wear leveling (distribute writes equally to all blocks, to avoid fast failures of a “hot” block)

FTL is typically implemented in hardware in the SSD, but is implemented in software for some SSDs.

FTL Approach #1: Direct Mapping

Have a 1-1 correspondence between logical pages and physical pages.

• Reading a page is straightforward
• Writing a page is expensive:
  1. Read the entire physical block into memory
  2. Update the relevant page in the in-memory block
  3. Erase the entire physical block
  4. Program the entire physical block using the new block value

Problems with direct mapping:

• Write amplification — writing a single page requires reading and writing an entire block (slow)
• Poor reliability — if the same logical block is repeatedly written, its physical block will quickly fail
• Data loss risk — if failure happens between erase and rewrite, data is lost

FTL Approach #2: Log-Based Mapping

Basic idea: treat the physical blocks like a log.

• Write data in each page-to-write to the end of the log
• Maintain a logical-to-physical mapping between logical pages and their corresponding physical pages in the SSD

How a write works:

1. If the log head is at the start of a new block, erase that block
2. Program the page at the log head with the new data
3. Update the logical-to-physical map
4. Advance the log head

Example

Writing logical page 92 with data w0: erase block 0, program physical page 0 with w0, map 92 ⇒ 0, advance log head. Writing logical page 17 with data w1: program physical page 1 with w1, map 17 ⇒ 1, advance log head. No erase needed since we are still in the same block.

Advantages over direct mapping:

• Avoids expensive read-modify-write behavior
• Better wear leveling: writes get spread across pages, even if there is spatial locality in writes at the logical level

Garbage Collection

When a logical page is overwritten, the old physical page becomes garbage (stale data). Over time, garbage pages accumulate.

Challenge:

• Overwritten pages take up space
• May share a block with valid data

Solution: garbage collection

1. Read valid data from a block (ignore garbage)
2. Write valid data to end of log
3. Update the logical-to-physical map
4. Erase the old block — it is now free space

Garbage collection requires extra read+write traffic.

Overprovisioning

Overprovisioning makes GC less painful:

• SSD exposes a logical page space that is smaller than the physical page space
• By keeping extra “hidden” pages around, the SSD tries to defer GC to a background task (removing GC from the critical path of a write)

The SSD will occasionally shuffle live (non-garbage) blocks that never get overwritten — this enforces wear leveling.

SSDs vs. Hard Drives (Throughput)

Lecture Note

SSDs dominate in random reads/writes and sequential performance. But hard drives are ~10x cheaper per storage bit. Note how the hard drive has essentially the same throughput for random and sequential access — it is bottlenecked by mechanical seek/rotation.

SSD Summary

• Most local storage will be solid state — faster, cheap enough
• Best for fast random access
• Relies on FTL for fast writes
  • Sequential and random writes have closer cost (unlike hard disks)
  • Garbage collection is more expensive with random writes

Related

• Navigation: ← Lecture 15
• Exam: Midterm 2
• Course: CS 537 Map of Content