IOMMU groups: read them before you pass anything through.

What is an IOMMU group?

An IOMMU (Intel VT-d, AMD-Vi or an Arm SMMU) translates the addresses a device uses for DMA, so the device can only reach memory it has been given. Linux can only enforce that boundary between devices whose transactions actually pass through the IOMMU. If two functions can reach each other directly, for example by peer-to-peer traffic below a switch or inside a multi-function card, the kernel cannot keep them apart. It puts them in the same IOMMU group.

The kernel’s VFIO documentation defines a group as “a set of devices which is isolatable from all other devices in the system” and makes it the unit of ownership: a VM gets a whole group or none of it. Linux kernel: VFIO groups, devices and IOMMUs.

Your IOMMU groups are a wiring diagram, not a setting

The IOMMU can only police traffic that reaches it. If two devices sit below the same switch or chipset port, a request from one to the other can turn around before it ever gets to the IOMMU. Linux cannot stop what it cannot see, so it draws one group around both devices.

Access Control Services (ACS) is the PCIe feature that closes that gap: it lets a port force peer-to-peer requests up through the IOMMU. Where a root port, switch port or multi-function device does not provide ACS (and no kernel quirk covers it), Linux has to assume the devices below it can reach each other. That is why slots wired to a consumer chipset often land in one large group.

So read your group listing as a map of how the board is wired. A big shared group is not a misconfigured kernel; it is the platform telling you which devices it cannot keep apart. That is why the fixes that work are physical or firmware-level: a different slot, a different board, or firmware that reports ACS. A kernel option that makes the group look smaller changes the map, not the wiring.

Is the IOMMU enabled?

No groups exist until the IOMMU is enabled. Turn on VT-d (Intel) or IOMMU / AMD-Vi (AMD) in firmware first. AMD hosts enable the IOMMU by default. On Intel the default depends on how the kernel was built: Proxmox documents it as on by default from its 6.8 kernels, and intel_iommu=on as needed on older ones. Proxmox VE admin guide source: PCI(e) passthrough.

cat /proc/cmdline
sudo dmesg | grep -e DMAR -e IOMMU -e AMD-Vi
ls /sys/kernel/iommu_groups/

An empty /sys/kernel/iommu_groups/ means no groups were created in this boot. Log wording varies by platform and kernel, so treat the dmesg lines as confirmation rather than a checklist. Linux ABI: /sys/kernel/iommu_groups.

Does iommu=pt change my groups?

No. The kernel parameter documentation describes iommu=pt as equivalent to iommu.passthrough=1: devices the host keeps use untranslated DMA by default. It changes the host’s DMA mode, not which devices share a group. Proxmox notes that it may improve performance where supported. Linux kernel: intel_iommu=, iommu=pt and iommu.passthrough= parameters.

How do I list IOMMU groups?

This read-only loop prints every group and the functions in it. It is the same command the IOMMU group checker expects.

for g in $(find /sys/kernel/iommu_groups/* -maxdepth 0 -type d | sort -V); do
  echo "IOMMU Group ${g##*/}:"
  for d in $g/devices/*; do echo -e "\t$(lspci -nns ${d##*/})"; done
done

For one device, read its group link directly. With -vv, lspci also prints an IOMMU group: line when the kernel exposes one. pciutils source: Control, Status, Region and IOMMU group lines.

# Replace 0000:01:00.0 with your device address.
readlink /sys/bus/pci/devices/0000:01:00.0/iommu_group
ls /sys/bus/pci/devices/0000:01:00.0/iommu_group/devices/
sudo lspci -vv -s 0000:01:00.0 | grep "IOMMU group"

On Proxmox VE, pvesh get /nodes/{nodename}/hardware/pci --pci-class-blacklist "" (with your node name) lists every device with an iommugroup column. Proxmox VE admin guide source: PCI(e) passthrough.

Annotated example: reading a group listing

# Synthetic listing for illustration: not captured from hardware.
IOMMU Group 0:
	00:00.0 Host bridge [0600]: Synthetic Header7F demo host bridge [1022:14d8]
IOMMU Group 1:
	00:01.0 Host bridge [0600]: Synthetic Header7F demo dummy bridge [1022:14da]
	00:01.1 PCI bridge [0604]: Synthetic Header7F demo root port [1022:14db]
	01:00.0 VGA compatible controller [0300]: Synthetic demo GPU (not a hardware capture) [10de:2684] (rev a1)
	01:00.1 Audio device [0403]: Synthetic demo GPU HDMI audio [10de:22ba] (rev a1)
IOMMU Group 2:
	00:02.1 PCI bridge [0604]: Synthetic Header7F demo chipset port [1022:14db]
	02:00.0 USB controller [0c03]: Synthetic demo USB controller [1022:43f7] (rev 01)
	02:00.1 SATA controller [0106]: Synthetic demo SATA controller [1022:43f6] (rev 01)
	03:00.0 Ethernet controller [0200]: Synthetic demo 2.5GbE controller [8086:15f3] (rev 03)
	04:00.0 VGA compatible controller [0300]: Synthetic demo second GPU [1002:744c] (rev c8)
IOMMU Group 3:
	05:00.0 Non-Volatile memory controller [0108]: Synthetic demo NVMe drive [144d:a80c]

The bracketed class code is the quick filter: [06xx] is a bridge, which VFIO does not need to own. Every other class is an endpoint, and VFIO must own all endpoints in the group before the group can be used. Proxmox documents sharing a group with the device’s own functions, its root port or a bridge as acceptable. Proxmox VE admin guide source: PCI(e) passthrough.

Group not clean? Triage in this order

1. Confirm groups exist. If /sys/kernel/iommu_groups/ is empty, fix the firmware and kernel IOMMU settings first. Nothing below applies yet.
2. List every function of your device. Functions share the bus and device number and differ after the dot: 01:00.0, 01:00.1. A GPU usually has an HDMI/DisplayPort audio function, and some cards expose further functions such as a USB controller. All of them belong to the guest.
3. Classify everything else in the group. Bridges and root ports ([06xx]) can stay with the host. Any other endpoint blocks a clean assignment.
4. Find where the slot is wired. Read the motherboard manual’s block diagram or lane table. Slots on CPU lanes often sit behind their own root port; chipset slots usually share one upstream link. Move the card only with the system powered off.
5. Check firmware. Look for IOMMU, ACS or PCIe options and the board maker’s current firmware. Save the group listing before and after any firmware change, because grouping follows what the platform reports.
6. Decide whether to pass the whole group. If the host can do without every other endpoint in the group, assigning them all is a valid, isolated configuration.
7. Treat the ACS override as the last option. Read the next section before using it.
8. Re-check, then bind. Re-run the listing, check the device in the IOMMU group checker, then bind its functions to vfio-pci.

What does the ACS override actually do?

pcie_acs_override is not in the mainline kernel parameter documentation. It comes from an out-of-tree patch that some kernels carry. The Proxmox wiki documents pcie_acs_override=downstream as a last option that is not without risk. Proxmox VE wiki: PCI passthrough, GPU notes and ACS override.

The patch makes the kernel treat devices as isolated where the hardware has not said it isolates them. The groups get smaller; the hardware does not change. If peer-to-peer traffic between those devices can still bypass the IOMMU, a passed-through device could reach a device the host or another VM still owns. That may be an acceptable trade on a single-user lab machine. It is a poor one on a host that runs untrusted guests or holds data you need to protect. If you apply it, record that and why.

After the group is clean: bind and verify

Bind every endpoint function in the group to vfio-pci. On Proxmox, the host tries to release the device itself when the VM starts; static binding is covered in the Proxmox GPU passthrough guide. Verify on the host:

# Example: all functions of device 01:00.
lspci -nnk -s 01:00
# Expect "Kernel driver in use: vfio-pci" under each endpoint function.

If QEMU reports group N is not viable, at least one device in that group is still owned by a host driver; QEMU’s own hint asks you to bind all devices in the group to their vfio bus driver. QEMU source: the VFIO “group is not viable” error. The kernel’s VFIO documentation describes the same rule. Linux kernel: VFIO groups, devices and IOMMUs.

A clean group is necessary, not sufficient. Reset behaviour and the host’s use of the boot GPU can still stop a VM. If the card becomes unreadable after the first VM shutdown, read unknown PCI header type 127 in passthrough.

Questions

Is it fine if my GPU shares an IOMMU group with its root port?

Yes. Proxmox documents sharing a group with the device’s own functions, its root port or a PCI(e) bridge as acceptable. Bridges stay on their host driver; only the endpoints in the group must be owned by VFIO.

Why is everything in one IOMMU group?

Usually because the IOMMU is on but the platform does not report ACS isolation between its ports, so Linux cannot separate the devices. Check firmware IOMMU and ACS options and the board’s lane wiring. If /sys/kernel/iommu_groups/ is empty instead, the IOMMU is off.

Do I need intel_iommu=on?

It depends on the kernel build. AMD enables its IOMMU by default. Proxmox documents the Intel IOMMU as on by default from its 6.8 kernels; older or differently configured kernels need intel_iommu=on. Check /sys/kernel/iommu_groups/ after boot rather than guessing.

Does iommu=pt change IOMMU groups?

No. It makes untranslated DMA the default for devices the host keeps (equivalent to iommu.passthrough=1). Group membership comes from the hardware topology and ACS, not from this parameter.

Can a firmware update change my IOMMU groups?

It can, because groups follow the topology and ACS capabilities the platform exposes. Save the group listing before updating and compare it afterwards, so a passthrough setup that stops working has an obvious first check.

Is the ACS override safe?

It removes the kernel’s isolation check without adding isolation. Devices the kernel now reports as separate may still reach each other by peer-to-peer DMA. Use it only where you accept that risk, and never as a default.