libkrun — vCPU Runtime

vCPU execution loop vstate.rs:1571

Guest vCPU runs (KVM_RUN / hv_vcpu_run) run_emulation:1428

↓ VM exit

MMIO Exit

Queue Kick

HLT / Shutdown

Signal / Intr

KVM_EXIT_MMIO vstate.rs:1502 — the guest read or wrote to a guest-physical address in the MMIO window.

Handler path: VcpuExit::MmioRead/Write → MMIODeviceManager.dispatch_mmio(addr, data) → Bus.get_device(addr) bus.rs:103 → MmioTransport.read/write() mmio.rs:377 → device register access.

This is how the guest driver negotiates features, submits queue kicks, and reads device status — all via MMIO reads/writes routed through this exit.

Queue notification (EventFd) — the guest wrote to the QUEUE_NOTIFY MMIO register, which signals an EventFd. The EventManager wakes the device worker thread.

Flow: guest write → MmioTransport::write(QUEUE_NOTIFY) → queue_evts[n].write(1) → EventManager wakes → device processes descriptors queue.rs:452 → marks used → interrupt_evt.write(1) → inject IRQ into vCPU.

KVM_EXIT_HLT / KVM_EXIT_SHUTDOWN vstate.rs:1514 — the guest executed HLT (x86) or a shutdown hypercall. The vCPU thread exits its loop, signals other vCPUs, and tears down the VM.

On ARM: PSCI CPU_OFF / SYSTEM_OFF hypercalls route to VcpuExit::SystemEvent vstate.rs:1532. The VMM sends a shutdown signal to all other vCPU threads.

Signal / KVM_EXIT_INTR — the vCPU was interrupted by a Unix signal (typically SIGUSR1 sent from the EventManager or another thread to wake the vCPU). The vCPU returns from KVM_RUN, processes pending events (injected IRQs, device work), then re-enters the guest.

Thread model

vCPU thread ×N

Vcpu::run() loop vstate.rs:1571

One OS thread per virtual CPU. Calls vcpu_fd.run() in a tight loop. Each iteration either completes without an exit (ideal case) or returns with a VcpuExit reason that must be handled before re-entering.

Event thread ×1

EventManager epoll loop

Main thread after build_microvm. Runs epoll_wait over all registered EventFds — virtio queue kicks, device interrupt signals, and process signals. Dispatches each event to the appropriate handler closure.

Device threads

Per-device worker (block, vsock, net)

Some devices (block, vsock muxer muxer.rs:99) run their own threads to perform I/O without blocking the event loop. They communicate back via EventFds that the EventManager polls.

VM exit type reference run_emulation:1428

Exit reason	What happened	Handler action
KVM_EXIT_MMIO :1502	Guest accessed a guest-physical address outside RAM	Route to MmioTransport via Bus; complete read/write; resume
KVM_EXIT_IO :1471	Guest executed IN/OUT port instruction (x86-64 only)	Route to legacy I/O bus (serial, CMOS, i8042); resume
KVM_EXIT_INTR	KVM_RUN interrupted by a signal to the vCPU thread	Check for pending work; re-enter KVM_RUN
KVM_EXIT_HLT :1514	Guest executed HLT (x86) with no pending IRQ	Wait for interrupt kick before resuming
KVM_EXIT_SHUTDOWN :1518	Guest triple-faulted or issued shutdown (e.g. ACPI off)	Signal all vCPU threads to stop; EventManager exits
KVM_EXIT_SYSTEM_EVENT :1532	Guest issued PSCI reboot/poweroff (ARM)	Parse event type; trigger VM shutdown or reset
KVM_EXIT_FAIL_ENTRY :1524	vCPU could not enter the guest (bad state)	Log hardware_entry_failure_reason; abort
HV_EXIT_REASON_VTIMER macos:358	Virtual timer fired (macOS HVF, ARM)	Inject virtual IRQ; resume

Interrupt injection path

Device completes I/O └─ interrupt_evt.write(1) // EventFd signal └─ EventManager wakes └─ InterruptTransport::signal() └─ Linux: irqfd → KVM injects IRQ atomically macOS: hv_vcpu_interrupt() call macos/vstate.rs:438 └─ vCPU picks up IRQ on next entry └─ Guest IRQ handler runs └─ virtio driver reads used ring queue.rs:502