Repository navigation
Releases: amd/MxGPU-Virtualization
Release list
GIM 9.3.0.K Release Note
We are excited to announce a new open-source release of our GIM SR-IOV Driver. Along with regular bug fixes and improvements, this release is aligned with ROCm 10.1.
New in this release:
- Ubuntu 26.04 support - added as a host and guest OS on MI210X, MI300X, MI325X, MI350X, and MI355X
- Ubuntu 26.04 and RHEL 10.2 support - added Ubuntu 26.04 as a host OS and Ubuntu 26.04 and RHEL 10.2 as a guest OS on Radeon V710
- VF power-cap control - Enables power-cap management for virtual functions on MI210X
Hardware / Features Supported for this Release
| AMD GPU | Host OS | Guest OS | VF per GPU - Compute Mode - Memory Mode | Infinity Fabric (XGMI) Configuration | ROCm / Graphics Version |
|---|---|---|---|---|---|
| AMD Instinct MI355X / AMD Instinct MI350X | Ubuntu 26.04 Kernel 7.0 | Ubuntu 26.04 | 1 - SPX - NPS1 | Between 8 GPUs | ROCm 10.1 |
| Ubuntu 24.04 Kernel 6.14 | Ubuntu 24.04 | 1 - SPX - NPS1 2* - DPX - NPS2 8* - CPX - NPS2 | |||
| RHEL 10.0 | 1 - SPX - NPS1 | ||||
| RHEL 9.6 | 1 - SPX - NPS1 | ||||
| AMD Instinct MI325X | Ubuntu 26.04 Kernel 7.0 | Ubuntu 26.04 | 1 - SPX - NPS1 | Between 8 GPUs | ROCm 10.1 |
| Ubuntu 22.04 Kernel 6.8 | Ubuntu 22.04 | 1 - SPX - NPS1 | |||
| RHEL 10.2 Kernel 6.12.0-211 | RHEL 10.2 | 1 - SPX - NPS1 | |||
| RHEL 9.4 Kernel 5.14.0-427 | RHEL 9.4 | 1 - SPX - NPS1 | |||
| Ubuntu 26.04 | 1 - SPX - NPS1 | ||||
| AMD Instinct MI300X | Ubuntu 26.04 Kernel 7.0 | Ubuntu 26.04 | 1 - SPX - NPS1 | Between 8 GPUs | ROCm 10.1 |
| Ubuntu 24.04 Kernel 6.14 | Ubuntu 24.04 | 1 - SPX - NPS1 8* - CPX - NPS4 | |||
| Ubuntu 22.04 Kernel 6.8 | Ubuntu 22.04 | 1 - SPX - NPS1 | |||
| RHEL 10.2 Kernel 6.12.0-211 | RHEL 10.2 | 1 - SPX - NPS1 | |||
| RHEL 9.4 Kernel 5.14.0-427 | RHEL 9.4 | 1 - SPX - NPS1 8* - CPX - NPS4 | |||
| Ubuntu 24.04 | 1 - SPX - NPS1 8* - CPX - NPS4 | ||||
| AMD Instinct MI210X | Ubuntu 26.04 Kernel 7.0 | Ubuntu 26.04 | 1 | Between 4 / 8 GPUs | ROCm 10.1 |
| RHEL 9.4 Kernel 5.14.0-427 | RHEL 9.4 | 1 | |||
| Ubuntu 22.04 | 1 | ||||
| AMD Radeon PRO V710 | Ubuntu 26.04 Kernel 7.0 | ROCm and Graphics: Ubuntu 26.04 Ubuntu 24.04 RHEL 10.2 RHEL 9.6 |
1 to 12 | N/A | ROCm 10.1 amdgpu 6.4.2.2 |
| Ubuntu 24.04 Kernel 6.14 | ROCm and Graphics: Ubuntu 24.04 RHEL 10.2 RHEL 9.6 |
1 to 12 |
Partition Mode Support
| Number of VFs per GPU | Dynamic Compute Partitioning | NPS1 | NPS2 | NPS4 |
|---|---|---|---|---|
| 1 | SPX (Default) | MI300X, MI325X, MI35XX | ||
| 1 | DPX | MI35XX | ||
| 1 | CPX | MI35XX | ||
| 2* | DPX | MI35XX | ||
| 8* | CPX | MI35XX | MI300X |
For the full list of supported partition modes and the relationship between partition mode and number of VFs per GPU, see the GPU Partitioning.
* Multi-VF Support requires a compatible firmware. Please refer to the ROCm Firmware Compatibility Table
Host AMD SMI Versions
- Library: 64.6.1
- Tool: 38.3.2
Please checkout out SMI doc for more details.
Getting Started
Please check out our User Guide for instructions on how to set up GIM and example configurations to run SR-IOV enabled VMs.
Known Issues
- On VMs with multiple VF assigned, detaching the VF (e.g. with virsh detach-device) while a workload is running will cause the guest kernel to refuse further GPU work. The VM needs to be restarted before GPU work can be issued again. This issue is limited to SPX/NPS1 configuration only.
- MI350X: 1 VF mode, there is a small chance VF FLRs will fail due to an SMU timeout if a reset is triggered concurrently with an active GFX workload. The VF FLR will be promoted to a whole GPU reset in these cases. This reset may interrupt workloads running on other VFs in a multi-tenancy setup.
- Latest Ubuntu Kernel version 6.8.0-136-generic has a regression causing boot times to increase. This regression is being reverted, until then please use Ubuntu Kernel version 6.8.0-90-generic.
GIM 9.2.0.K Release Note
We are excited to announce a new open-source release of our GIM SR-IOV Driver. Along with regular bug fixes and improvements, this release is aligned with ROCm 10.0,
- AMD-SMI/GIM reliability — Bug fixes: GPU-reset event reporting and power-cap data behavior,
- RHEL 10.2 support — New OS enablement for MI325X and MI300X host/guest combinations.
Hardware / Features Supported for this Release
| Hardware | Host OS | Guest OS | ROCm/Graphics version | Number of VFs per GPU | Infinity Fabric (XGMI) Configuration |
|---|---|---|---|---|---|
| AMD Instinct MI355X / AMD Instinct MI350X | Ubuntu 24.04 Kernel 6.14 | RHEL 9.6, RHEL 10.0, Ubuntu 24.04 | ROCm 10.0 | 1, 2*, or 8* | between 8 GPUs |
| AMD Instinct MI325X | Ubuntu 22.04 Kernel 6.8 | Ubuntu 22.04 | ROCm 10.0 | 1 | between 8 GPUs |
| RHEL 10.2** kernel 6.12.0-211 | RHEL 10.2** | ROCm 10.0 | 1 | between 8 GPUs | |
| AMD Instinct MI300X | Ubuntu 22.04 Kernel 6.8, Ubuntu 24.04 Kernel 6.14 | Ubuntu 22.04, Ubuntu 24.04 | ROCm 10.0 | 1 or 8* | between 8 GPUs |
| RHEL 9.4 Kernel 5.14.0-427, RHEL 10.2** kernel 6.12.0-211 |
RHEL 9.4, RHEL 10.2**, Ubuntu 24.04 |
ROCm 10.0 | 1 or 8* | between 8 GPUs | |
| AMD Instinct MI210X | RHEL 9.4 Kernel 5.14.0-427 | RHEL 9.4, Ubuntu 22.04 | ROCm 10.0 | 1 | between 4/8 GPUs |
| AMD Radeon PRO V710 | Ubuntu 24.04 Kernel 6.14 | ROCm: RHEL 9.6, Ubuntu 24.04 Graphics: Ubuntu 24.04 |
ROCm 10.0 amdgpu 6.4.2.2 |
1 to 12 | N/A |
Partition Mode Support
| Number of VFs per GPU | Dynamic Compute Partitioning | NPS1 | NPS2 | NPS4 |
|---|---|---|---|---|
| 1 | SPX (Default) | MI300X, MI325X, MI35XX | ||
| 1 | DPX | MI35XX | ||
| 1 | CPX | MI35XX | ||
| 2* | DPX | MI35XX | ||
| 8* | CPX | MI35XX | MI300X |
For the full list of supported partition modes and the relationship between partition mode and number of VFs per GPU, see the GPU Partitioning.
* Multi-VF support requires a compatible firmware. Please refer to the ROCm Firmware Compatibility Table for the list of required firmware versions and supported configurations.
** RHEL 10.1 is EOLed and replaced by RHEL 10.2
Host AMD SMI Versions
- Library: 64.0.1
- Tool: 37.0.5
Please checkout out SMI doc for more details.
Getting Started
Please check out our User Guide for instructions on how to set up GIM and example configurations to run SR-IOV enabled VMs.
Known Issues
- On VMs with multiple VF assigned, detaching one of the VF (e.g. with virsh detach-device) while a workload is running will cause the guest kernel to refuse further GPU work. The VM need to be restarted before GPU work can be issued again.
- MI350X: In 1 VF mode, there is a small chance VF FLRs will fail due to an SMU timeout if a reset is triggered concurrently with an active GFX workload. The VF FLR will be promoted to a whole GPU reset in these cases. This reset may interrupt workloads running on other VFs in a multi-tenancy setup.
GIM 9.1.0.K Release Note
We are excited to announce a new open-source release of our GIM SR-IOV Driver. Along with regular bug fixes and improvements, this release provides support for ROCm 7.14,
- MI350/MI355 multiVF - enabled and optimized multi-VF partition modes including NPS2/DPX and NPS2/CPX configurations,
- expanded AMD-SMI host telemetry with UBB power data, thermal metrics per partition for MI35x, and violation status reporting,
- added AMD-SMI support for listing and monitoring Pensando and Broadcom NIC devices, and
- added AMD-SMI support for Go bindings.
Hardware / Features Supported for this Release
| Hardware | Host OS | Guest OS | ROCm/Graphics version | Number of VFs per GPU | Infinity Fabric (XGMI) Configuration |
|---|---|---|---|---|---|
| AMD Instinct MI355X / AMD Instinct MI350X | Ubuntu 24.04 Kernel 6.14 | RHEL 9.6, RHEL 10.0, Ubuntu 24.04 | ROCm 7.14 | 1, 2*, or 8* | between 8 GPUs |
| AMD Instinct MI325X | Ubuntu 22.04 Kernel 6.8 | Ubuntu 22.04 | ROCm 7.14 | 1 | between 8 GPUs |
| RHEL 10.1 Kernel 6.12.0-124 | RHEL 10.1 | ROCm 7.14 | 1 | between 8 GPUs | |
| AMD Instinct MI300X | Ubuntu 22.04 Kernel 6.8, Ubuntu 24.04 Kernel 6.14 | Ubuntu 22.04, Ubuntu 24.04 | ROCm 7.14 | 1 or 8* | between 8 GPUs |
| RHEL 9.4 Kernel 5.14.0-427 | RHEL 9.4, Ubuntu 24.04 | ROCm 7.14 | 1 or 8* | between 8 GPUs | |
| AMD Instinct MI210X | RHEL 9.4 Kernel 5.14.0-427 | RHEL 9.4, Ubuntu 22.04 | ROCm 7.14 | 1 | between 4/8 GPUs |
| AMD Radeon PRO V710 | Ubuntu 24.04 Kernel 6.14 | ROCm: RHEL 9.6, Ubuntu 24.04 Graphics: Ubuntu 24.04 |
ROCm 7.14 amdgpu 6.4.2.2 |
1, 4, 6, or 12 | N/A |
Partition Mode Support
| Number of VFs per GPU | Dynamic Compute Partitioning | NPS1 | NPS2 | NPS4 |
|---|---|---|---|---|
| 1 | SPX (Default) | MI300X, MI325X, MI35XX | ||
| 1 | DPX | MI35XX | ||
| 1 | CPX | MI35XX | ||
| 2* | DPX | MI35XX | ||
| 8* | CPX | MI35XX | MI300X |
For the full list of supported partition modes and the relationship between partition mode and number of VFs per GPU, see the GPU Partitioning.
* Multi-VF support requires a compatible firmware. Please refer to the ROCm Firmware Compatibility Table for the list of required firmware versions and supported configurations.
Host AMD SMI Versions
- Library: 62.0.5
- Tool: 36.1.2
Please checkout out SMI doc for more details.
Getting Started
Please check out our User Guide for instructions on how to set up GIM and example configurations to run SR-IOV enabled VMs.
Known Issues
- MI300X/MI350X: In configurations with 8 VFs per GPU, VF FLR may intermittently fail with "SMU FW not responding" or "SMU Timeout" errors on the host. This condition can result in a WGR on all GPUs. If the issue occurs, all active VMs must be restarted to restore normal operation.
GIM 9.0.0.K Release Note
We are excited to announce a new open-source release of our GIM SR-IOV Driver.
Along with regular bug fixes and improvements, this release provides support for ROCm 7.13 preview.
Hardware / Features Supported for this Release
| Hardware | Host OS | Guest OS | ROCm/Graphics version | Number of VFs per GPU | Infinity Fabric (XGMI) Configuration |
|---|---|---|---|---|---|
| AMD Instinct MI355X / AMD Instinct MI350X | Ubuntu 24.04 Kernel 6.14 | RHEL 9.6, RHEL 10.0, Ubuntu 24.04 | ROCm 7.13 | 1 | between 8 GPUs |
| AMD Instinct MI325X | Ubuntu 22.04 Kernel 6.8 | Ubuntu 22.04 | ROCm 7.13 | 1 | between 8 GPUs |
| RHEL 10.1 Kernel 6.12.0-124 | RHEL 10.1 | ROCm 7.13 | 1 | between 8 GPUs | |
| AMD Instinct MI300X | Ubuntu 22.04 Kernel 6.8, Ubuntu 24.04 Kernel 6.14 | Ubuntu 22.04, Ubuntu 24.04 | ROCm 7.13 | 1 or 8* | between 8 GPUs |
| RHEL 9.4 Kernel 5.14.0-427 | RHEL 9.4, Ubuntu 24.04 | ROCm 7.13 | 1 or 8* | between 8 GPUs | |
| AMD Instinct MI210X | RHEL 9.4 Kernel 5.14.0-427 | RHEL 9.4, Ubuntu 22.04 | ROCm 7.13 | 1 | between 4/8 GPUs |
| AMD Radeon PRO V710 | Ubuntu 24.04 Kernel 6.14 | ROCm: RHEL 9.6, Ubuntu 24.04 Graphics: Ubuntu 24.04 |
ROCm 7.13 (preview) amdgpu 6.4.2.2 |
1, 4, 6, or 12 | N/A |
Partition Mode Support
| Number of VFs per GPU | Dynamic Compute Partitioning | NPS1 | NPS2 | NPS4 |
|---|---|---|---|---|
| 1 | SPX (Default) | MI300X, MI325X, MI35XX | ||
| 1 | DPX | MI35XX | ||
| 1 | CPX | MI35XX | ||
| 8* | CPX | MI300X (Preview) |
For the full list of supported partition modes and the relationship between partition mode and number of VFs per GPU, see the GPU Partitioning.
* Multi-VF (8 VF) Support requires a compatible firmware. Please refer to the ROCm Firmware Compatibility Table for the list of required firmware versions and supported configurations.
Host AMD SMI Versions
- Library: 54.4.4
- Tool: 34.12.3
Please checkout out SMI doc for more details.
Getting Started
Please check out our User Guide for instructions on how to set up GIM and example configurations to run SR-IOV enabled VMs.
Known Issues
- MI300X: In MI300X multi-VF configurations (CPX + NPS4), a RAS uncorrectable error (UE) may occur when a VF Function Level Reset (FLR) is triggered while compute workloads are actively running across multiple virtual machines. The error originates in the GFX subsystem and results in a failed Mode-1 reset, requiring a full system power cycle to recover. This issue is observed specifically when a compute engine hang occurs during an active compute workload and a subsequent VF FLR is initiated for recovery. It does not reproduce with DMA-only workloads or when VF FLR is performed without an active compute workload. The issue is not observed in single-VF configurations.
- MI210X: Unloading the Guest driver on MI210X may produce kernel call trace. Note that this is just a warning and does not affect functionality.
GIM 8.7.1.K Release Note
We are excited to announce a new open-source release of our GIM SR-IOV Driver.
Along with regular bug fixes and improvements, this release expands KVM SR-IOV virtualization support for MI325X on RHEL 10 and enables ROCm profiler tool support within KVM SR-IOV virtualized (Guest) GPU environments..
Hardware / Features Supported for this Release
| Hardware | Host OS | Guest OS | ROCm version | Number of VFs per GPU | Infinity Fabric (XGMI) Configuration |
|---|---|---|---|---|---|
| AMD Instinct MI355X | Ubuntu 24.04 Kernel 6.14 | RHEL 9.6, RHEL 10.0, Ubuntu 24.04 | ROCm 7.2.1 | 1 | between 8 GPUs |
| AMD Instinct MI350X | Ubuntu 24.04 Kernel 6.14 | RHEL 9.6, RHEL 10.0, Ubuntu 24.04 | ROCm 7.2.1 | 1 | between 8 GPUs |
| AMD Instinct MI325X | Ubuntu 22.04 Kernel 6.8 | Ubuntu 22.04 | ROCm 7.2.1 | 1 | between 8 GPUs |
| AMD Instinct MI325X | RHEL 10.1 Kernel 6.12.0-124 | RHEL 10.1 | ROCm 7.2.1 | 1 | between 8 GPUs |
| AMD Instinct MI300X | Ubuntu 22.04 Kernel 6.8 | Ubuntu 22.04 | ROCm 7.2.1 | 1 or 8* | between 8 GPUs |
| AMD Instinct MI300X | RHEL 9.4 Kernel 5.14.0-427 | RHEL 9.4, Ubuntu 24.04 | ROCm 7.2.1 | 1 or 8* | between 8 GPUs |
| AMD Instinct MI210X | RHEL 9.4 Kernel 5.14.0-427 | RHEL 9.4, Ubuntu 22.04 | ROCm 7.2.1 | 1 | between 4/8 GPUs |
| AMD Radeon PRO V710 | Ubuntu 24.04 Kernel 6.14 | Ubuntu 24.04 | ROCm 7.2.1 | 1 or 4 or 12 | N/A |
Partition Mode Support
| Number of VFs per GPU | Dynamic Compute Partitioning | NPS1 | NPS2 | NPS4 |
|---|---|---|---|---|
| 1 | SPX (Default) | MI300X, MI325X, MI35XX | ||
| 1 | DPX | MI35XX | ||
| 1 | CPX | MI35XX (Preview) | ||
| 8* | CPX | MI300X (Preview) |
For the full list of supported partition modes and the relationship between partition mode and number of VFs per GPU, see the GPU Partitioning.
* Multi-VF (8 VF) Support requires a compatible firmware BKC bundle and ROCm which will be released in coming months.
Please refer to the ROCm Firmware Compatibility Table for the list of required firmware versions and supported configurations.
Host AMD SMI Versions
- Library: 51.0.0
- Tool: 34.1.1
Please checkout out SMI doc for more details.
Getting Started
Please check out our User Guide for instructions on how to set up GIM and example configurations to run SR-IOV enabled VMs.
Known Issues
- MI300: MultiVF configuration for MI300X requires a compatible firmware BKC bundle and ROCm which will be released in coming months. Please refer to the ROCm compatibility matrix for supported version
- MI210: Unloading the Guest driver on MI210 may produce kernel call trace. Note that this is just a warning and not a kernel panic. Guest VM reboot is required to restore the operation
- Navi32: AMDSMI CLI --gpureset command on Nv32 doesn't reset all the GPUs like the output is saying, but just GPU 0.
GIM 8.7.0.K Release Note
We are excited to announce a new open-source release of our GIM SR-IOV Driver.
This release has regular bug fixes and improvements, including MI350/MI355 Node Power Management for multi-GPU nodes (for more info see ROCm release notes) and optimization for MI350/MI355 NPS2+DPX.
Hardware / Features Supported for this Release
| Hardware | Host OS | Guest OS | ROCm version | Number of VFs per GPU | Infinity Fabric (XGMI) Configuration |
|---|---|---|---|---|---|
| AMD Instinct MI355X | Ubuntu 24.04 Kernel 6.14 | RHEL 9.6, RHEL 10.0, Ubuntu 24.04 | ROCm 7.2 | 1 | between 8 GPUs |
| AMD Instinct MI350X | Ubuntu 24.04 Kernel 6.14 | RHEL 9.6, RHEL 10.0, Ubuntu 24.04 | ROCm 7.2 | 1 | between 8 GPUs |
| AMD Instinct MI325X | Ubuntu 22.04 Kernel 6.8 | Ubuntu 22.04 | ROCm 7.2 | 1 | between 8 GPUs |
| AMD Instinct MI300X | Ubuntu 22.04 Kernel 6.8 | Ubuntu 22.04 | ROCm 7.2* | 1 or 8* | between 8 GPUs |
| AMD Instinct MI300X | RHEL 9.4 Kernel 5.14.0-427 | RHEL 9.4, Ubuntu 24.04 | ROCm 7.2* | 1 or 8* | between 8 GPUs |
| AMD Instinct MI210X | RHEL 9.4 Kernel 5.14.0-427 | RHEL 9.4, Ubuntu 22.04 | ROCm 7.2 | 1 | between 4/8 GPUs |
| AMD Radeon PRO V710 | Ubuntu 24.04 Kernel 6.14 | Ubuntu 24.04 | ROCm 7.2 | 1 or 4 or 12 | N/A |
Partition Mode Support
| Number of VFs per GPU | Dynamic Compute Partitioning | NPS1 | NPS2 | NPS4 |
|---|---|---|---|---|
| 1 | SPX (Default) | MI300X, MI325X, MI35XX | ||
| 1 | DPX | MI35XX | ||
| 1 | CPX | MI35XX (Preview) | ||
| 8* | CPX | MI300X (Preview) |
For the full list of supported partition modes and the relationship between partition mode and number of VFs per GPU, see the GPU Partitioning.
* Multi-VF (8 VF) Support requires a compatible firmware BKC bundle for the GPU.
Please refer to the ROCm Firmware Compatibility Table for the list of required firmware versions and supported configurations.
Host AMD SMI Versions
- Library: 51.0.0
- Tool: 34.1.1
Please checkout out SMI doc for more details.
Getting Started
Please check out our User Guide for instructions on how to set up GIM and example configurations to run SR-IOV enabled VMs.
Known Issues
- MultiVF configuration for MI300X requires specific firmware version. please refer to the ROCm compatibility matrix for supported version
- MI210: Unloading the Guest driver on MI210 may produce kernel call trace. Note that this is just a warning and not a kernel panic. Guest VM reboot is required to restore the operation
GIM 8.6.0.K Release Note
We are excited to announce a new open-source release of our GIM SR-IOV Driver.
This release has regular bug fixes and improvements.
Hardware / Features Supported for this Release
| Hardware | Host OS | Guest OS | ROCm version | Number of VFs per GPU | Infinity Fabric (XGMI) Configuration |
|---|---|---|---|---|---|
| AMD Instinct MI355X | Ubuntu 24.04 | RHEL 9.6, RHEL 10, Ubuntu 24.04 | ROCm 7.1.1 | 1 | between 8 GPUs |
| AMD Instinct MI350X | Ubuntu 24.04 | RHEL 9.6, RHEL 10, Ubuntu 24.04 | ROCm 7.1.1 | 1 | between 8 GPUs |
| AMD Instinct MI325X | Ubuntu 22.04 | Ubuntu 22.04 | ROCm 7.1.1 | 1 | between 8 GPUs |
| AMD Instinct MI300X | Ubuntu 22.04 | Ubuntu 22.04 | ROCm 7.1.1* | 1 or 8* | between 8 GPUs |
| AMD Instinct MI300X | RHEL 9.4 | RHEL 9.4, Ubuntu 24.04 | ROCm 7.1.1* | 1 or 8* | between 8 GPUs |
| AMD Instinct MI210X | RHEL 9.4 | RHEL 9.4, Ubuntu 22.04 | ROCm 7.1.1 | 1 | between 4/8 GPUs |
| AMD Radeon PRO V710 | Ubuntu 24.04 | Ubuntu 24.04 | ROCm 7.1.1 | 1 or 4 or 12 | N/A |
Partition Mode Support
| Number of VFs per GPU | Dynamic Compute Partitioning | NPS1 | NPS2 | NPS4 |
|---|---|---|---|---|
| 1 | SPX (Default) | MI300X, MI325X, MI35XX | ||
| 1 | DPX | MI35XX | ||
| 1 | CPX | MI35XX (Preview) | ||
| 8* | CPX | MI300X (Preview) |
For the full list of supported partition modes and the relationship between partition mode and number of VFs per GPU, see the GPU Partitioning.
* Multi-VF (8 VF) Support requires a compatible firmware BKC bundle for the GPU.
Please refer to the ROCm Firmware Compatibility Table for the list of required firmware versions and supported configurations.
Host AMD SMI Versions
- Library: 50.2.2
- Tool: 32.4.1
Please checkout out SMI doc for more details.
Getting Started
Please check out our User Guide for instructions on how to set up GIM and example configurations to run SR-IOV enabled VMs.
Known Issues
- When using the configuration of 64 ROCM devices in 1 VM (aka CPX mode with 1 VF) on a MI350 platform, we would see a hang in workloads which use SDMA engines
- MultiVF configuration for MI300X requires specific firmware version. please refer to the ROCm compatibility matrix for supported version
- MI210: Unloading the Guest driver on MI210 may produce kernel call trace. Note that this is just a warning and not a kernel panic. Guest VM reboot is required to restore the operation
- SMI CLI topology --link_type command returns wrong value for the first BDF
- SMI CLI static --vram command returns 0 for the bit width value
- Some BDF values returned from SMI get_link_topology_nearest APIs are wrong
GIM 8.5.0.K Release Note
We are excited to announce a new open-source release of our GIM SR-IOV Driver.
This release adds MI350 CPX support and SMI CLI command improvement on top of the regular update and bug fixes.
Hardware / Features Supported for this Release
| Hardware | Host OS | Guest OS | ROCm version | Number of VFs per GPU | Infinity Fabric (XGMI) Configuration |
|---|---|---|---|---|---|
| AMD Instinct MI355X | Ubuntu 24.04 | RHEL 9.6, RHEL 10, Ubuntu 24.04 | ROCm 7.1 | 1 | between 8 GPUs |
| AMD Instinct MI350X | Ubuntu 24.04 | RHEL 9.6, RHEL 10, Ubuntu 24.04 | ROCm 7.1 | 1 | between 8 GPUs |
| AMD Instinct MI325X | Ubuntu 22.04 | Ubuntu 22.04 | ROCm 7.1 | 1 | between 8 GPUs |
| AMD Instinct MI300X | Ubuntu 22.04 | Ubuntu 22.04 | ROCm 7.1 | 1 or 8* | between 8 GPUs |
| AMD Instinct MI300X | RHEL 9.4 | RHEL 9.4 | ROCm 7.1 | 1 or 8* | between 8 GPUs |
| AMD Instinct MI210X | RHEL 9.4 | RHEL 9.4, Ubuntu 22.04 | ROCm 7.1 | 1 | between 4/8 GPUs |
| AMD Radeon PRO V710 | Ubuntu 24.04 | Ubuntu 24.04 | ROCm 7.1 | 1 or 4 or 12 | N/A |
Partition Mode Support
| Number of VFs per GPU | Dynamic Compute Partitioning | NPS1 | NPS2 | NPS4 |
|---|---|---|---|---|
| 1 | SPX (Default) | MI300X, MI325X, MI35XX | ||
| 1 | DPX | MI35XX | ||
| 1 | CPX | MI35XX (Preview) | ||
| 8* | CPX | MI300X (Preview) |
For the full list of supported partition modes and the relationship between partition mode and number of VFs per GPU, see the GPU Partitioning.
* Multi-VF (8 VF) Support requires a compatible firmware BKC bundle for the GPU.
Please refer to the ROCm Firmware Compatibility Table for the list of required firmware versions and supported configurations.
Host AMD SMI Versions
- Library: 40.0.6
- Tool: 30.0.3
Please checkout out SMI doc for more details.
Getting Started
Please check out our User Guide for instructions on how to set up GIM and example configurations to run SR-IOV enabled VMs.
Known Issues
With NPS1/SPX and 8VMs (1VF per VM) on MI300, triggering reset via "/sys/kernel/debug/kfd/hang_hws" causes the GPU to become unusable. Rebooting the guest should solve the problem.
GIM 8.4.1.K Release Note
We are excited to announce a new open-source release of our GIM SR-IOV Driver.
This release adds VF FLR improvement on Navi32 as well as MI300 multi Vf on top of the regular update and bug fixes.
Hardware / Features Supported for this Release
| Hardware | Host OS | Guest OS | ROCm version | Number of VFs per GPU | Infinity Fabric (XGMI) Configuration |
|---|---|---|---|---|---|
| AMD Instinct MI355X | Ubuntu 24.04 | Ubuntu 24.04 | ROCm 7.0.2 | 1 | between 8 GPUs |
| AMD Instinct MI350X | Ubuntu 24.04 | Ubuntu 24.04 | ROCm 7.0.2 | 1 | between 8 GPUs |
| AMD Instinct MI325X | Ubuntu 22.04 | Ubuntu 22.04 | ROCm 7.0.2 | 1 | between 8 GPUs |
| AMD Instinct MI300X | Ubuntu 22.04 | Ubuntu 22.04 | ROCm 7.0.2 | 1 or 8* | between 8 GPUs |
| AMD Instinct MI300X | RHEL 9.4 | RHEL 9.4 | ROCm 7.0.2 | 1 or 8* | between 8 GPUs |
| AMD Instinct MI210X | RHEL 9.4 | RHEL 9.4, Ubuntu 22.04 | ROCm 7.0.2 | 1 | between 4/8 GPUs |
| AMD Radeon PRO V710 | Ubuntu 24.04 | Ubuntu 24.04 | ROCm 7.0.2 | 1 or 4 or 12 | N/A |
Partition Mode Support
| Number of VFs per GPU | Dynamic Compute Partitioning | NPS1 | NPS2 | NPS4 |
|---|---|---|---|---|
| 1 | SPX (Default) | MI300X, MI325X, MI35XX | ||
| 1 | DPX | MI35XX (preview) | ||
| 1 | CPX | |||
| 8* | CPX | MI300X (preview) |
For the full list of supported partition modes and the relationship between partition mode and number of VFs per GPU, see the GPU Partitioning.
* Multi-VF (8 VF) Support requires a compatible firmware BKC bundle for the GPU.
Please refer to the ROCm Firmware Compatibility Table for the list of required firmware versions and supported configurations.
Host AMD SMI Versions
- Library: 33.1.3
- Tool: 27.8.0
Since the last release, the library and tool will be automatically installed when installing the gim-dkms package—manual build step is no longer required.
If you manually copied AMD SMI tool/library to /usr, you may need to reset the ldconfig cache and/or remove the AMD SMI files under /usr.
Getting Started
Please check out our User Guide for instructions on how to set up GIM and example configurations to run SR-IOV enabled VMs.
Known Issues
- Ubuntu kernel
6.8.0-64and6.14.0-24and possibly other kernels on other distros have a regression.
Please update to a later kernel that contains the fix. - For V710, the VF active time reported by
amd-smiis not accurate.
GIM 8.4.0.K Release Note
We are excited to announce a new open-source release of our GIM SR-IOV Driver.
This release adds dynamic partitioning support as well as V710 validation on top of the regular update and bug fixes.
Hardware / Features Supported for this Release
| Hardware | Host OS | Guest OS | ROCm version | Number of VFs per GPU | Infinity Fabric (XGMI) Configuration |
|---|---|---|---|---|---|
| AMD Instinct MI355X | Ubuntu 24.04 | Ubuntu 24.04 | ROCm 7.0 | 1 | between 2/4/8 GPUs |
| AMD Instinct MI350X | Ubuntu 24.04 | Ubuntu 24.04 | ROCm 7.0 | 1 | between 2/4/8 GPUs |
| AMD Instinct MI325X | Ubuntu 22.04 | Ubuntu 22.04 | ROCm 7.0 | 1 | between 8 GPUs |
| AMD Instinct MI300X | Ubuntu 22.04 | Ubuntu 22.04 | ROCm 7.0 | 1 | between 2/4/8 GPUs |
| AMD Instinct MI300X | RHEL 9.4 | RHEL 9.4 | ROCm 7.0 | 1 | between 2/4/8 GPUs |
| AMD Instinct MI210X | RHEL 9.4 | RHEL 9.5, Ubuntu 22.04 | ROCm 7.0 | 1 | between 4/8 GPUs |
| AMD Radeon PRO V710 | Ubuntu 24.04 | Ubuntu 24.04 | ROCm 7.0 | 1 / 4 / 12 | N/A |
Partition Mode Support
| Partition Mode | SPX | DPX | QPX | CPX |
|---|---|---|---|---|
| NPS1 | MI35XX, MI325X, MI300X | |||
| NPS2 | MI35XX (preview) | |||
| NPS4 | MI325X, MI300X |
For the full list of supported partition modes and the relationship between partition mode and number of VFs per GPU, see the GPU Partitioning.
Host AMD SMI Versions
- Library: 33.1.3
- Tool: 27.8.0
Since the last release, the library and tool will be automatically installed when installing the gim-dkms package—manual build step is no longer required.
If you manually copied AMD SMI tool/library to /usr, you may need to reset the ldconfig cache and/or remove the AMD SMI files under /usr.
Getting Started
Please check out our User Guide for instructions on how to set up GIM and example configurations to run SR-IOV enabled VMs.
Known Issues
- Ubuntu kernel
6.8.0-64and6.14.0-24and possibly other kernels on other distros have a regression.
Please update to a later kernel that contains the fix. - For V710, the VF active time reported by
amd-smiis not accurate.