Repository navigation
[VFIO] Chapter 3: Section: 3 Hot-plug/unplug - #6356
Open
ShadowCurse wants to merge 7 commits into
Open
ShadowCurse wants to merge 7 commits into
ShadowCurse wants to merge 7 commits into
Conversation
Codecov Report❌ Patch coverage is Please upload reports for the commit 885604a to get more accurate results. Additional details and impacted files@@ Coverage Diff @@
## feature/vfio #6356 +/- ##
================================================
- Coverage 82.72% 82.61% -0.11%
================================================
Files 284 284
Lines 33815 34002 +187
================================================
+ Hits 27972 28090 +118
- Misses 5843 5912 +69
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
ShadowCurse
force-pushed
the
vfio_part_3_3
branch
from
October 30, 2026 13:25
f4bf4d9 to
e2a0f26
Compare
For some reason `cargo fmt` from devctr does not re-flow these lines, but `cargo fmt` on the host does. Signed-off-by: Egor Lazarchuk <yegorlz@amazon.co.uk>
ShadowCurse
force-pushed
the
vfio_part_3_3
branch
from
November 5, 2026 12:15
e2a0f26 to
86c67e2
Compare
PCIe hot-plug needs an ACPI guest kernel, like the PCI hot-plug tests do, so pin the VFIO tests which will also run with hot-plugged devices. Actual tests will be added in the next commit. Signed-off-by: Egor Lazarchuk <yegorlz@amazon.co.uk>
ShadowCurse
force-pushed
the
vfio_part_3_3
branch
from
November 5, 2026 13:17
86c67e2 to
bc46f67
Compare
Wire up the code to allow hot-plugging of passthrough devices after VM boot. The API is same as for usual passthrough device addition. The device is placed behind a free PCIe root port and the guest is notified about it, so no manual bus rescan is needed. Adding devices with duplicated ids or host SBDFs is disallowed. Signed-off-by: Egor Lazarchuk <yegorlz@amazon.co.uk>
With VFIO device hot-plug support we need to add all syscalls needed for VFIO devices creation to the VMM thread. Signed-off-by: Egor Lazarchuk <yegorlz@amazon.co.uk>
Next commit will add VFIO hot-unplug support and we neet to take care of the KVM and VFIO container when last VFIO device/group is removed. When the last VFIO group is detached from the container, the kernel drops its DMA mappings and the container returns to the unset state. This means that on the next hot-plug we will need to map the guest memory again. The reason we cannot just drop the `Container` with the last device is that KVM allows only a single KVM_DEV_TYPE_VFIO device per VM and on 5.10 host kernel, dropping `Container` (which contains kvm vfio device) does not destroy this device. This causes consecutive hot-plug requests to fail since KVM will reject creating of a new kvm vfio device. Signed-off-by: Egor Lazarchuk <yegorlz@amazon.co.uk>
ShadowCurse
force-pushed
the
vfio_part_3_3
branch
from
November 5, 2026 16:36
bc46f67 to
183fc9d
Compare
Implement passthrough device removal and wire it up to the DELETE API. Like for virtio devices, only devices behind PCIe root ports can be removed, either gracefully after the guest acknowledges the removal or forcefully when the `force` flag is set. Once the last passthrough device is removed, the kernel releases its DMA mappings of the guest memory when the last group is detached from it, so a following hot-plug will DMA map guest memory again. Forced removal resets the device and unpins the guest memory, which can take over a second on some hosts, so exclude passthrough device API calls from the test framework API call duration check. Signed-off-by: Egor Lazarchuk <yegorlz@amazon.co.uk>
Passthrough and vhost-user devices write guest memory without going through KVM dirty tracking or the Firecracker dirty bitmap. While they are attached snapshots are refused, but once a hot-plugged device is removed a diff snapshot would miss the pages it wrote, silently corrupting the guest memory on restore. Refuse diff snapshots after such a device was hot-plugged until a full snapshot succeeds and captures all of the guest memory. Signed-off-by: Egor Lazarchuk <yegorlz@amazon.co.uk>
ShadowCurse
force-pushed
the
vfio_part_3_3
branch
from
November 5, 2026 16:39
183fc9d to
f5b069f
Compare
ShadowCurse
marked this pull request as ready for review
November 5, 2026 16:49
ShadowCurse
added
the
Status: Awaiting review
ShadowCurse
force-pushed
the
vfio_part_3_3
branch
from
November 5, 2026 17:00
f5b069f to
885604a
Compare
11 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Add support for the VFO hot-plug/unplug.
Additionally do some refactoring of the
PciDevicesandDeviceManagerto bring order to both old virtio and new vfio hot-plug/unplug code.Reason
Part of #5679
License Acceptance
By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.PR Checklist
tools/devtool checkbuild --allto verify that the PR passesbuild checks on all supported architectures.
tools/devtool checkstyle --no-clippyto verify that the PRpasses the automated style checks.
how they are solving the problem in a clear and encompassing way.
in the PR.
CHANGELOG.md.Runbook for Firecracker API changes.
integration tests.
TODO.rust-vmm.