I have an ASRock B650M Pro RS motherboard, and an Icy Dock NVMe hot-swap bay. As I have it set up, the NVMe bay is attached to the main x16 PCIe slot using a pass-through card and some SFF-8643 cables. This slot is attached directly to the CPU and is normally used for graphics cards, which I’m not using it for.
This setup requires 4x4 bifurcation on that slot, and that part works without any issues.
When I have drives plugged in when the machine boots, they are recognized and configured correctly. I can unplug drives after the machine boots, and use the normal Linux tricks to detach the drive. However, after re-installing drives in those slots, or installing drives in slots that did not have drives installed when the machine booted, rescanning the bus does not find the newly installed drives. And this makes some sense because, per lspci, all of the slots are -Hotplug -Surprise.
The BIOS for this motherboard does not have any PCIe hotplug options. The closest thing appears to be the Thunderbolt setting, which works in conjunction with ASRock’s Thunderbolt AIC. When enabled, it marks the x4 PCIe slot as +Hotplug and +Surprise per Linux lspci. Otherwise, as mentioned, none of the PCIe slots are marked as such. Per the manual, this slot is attached to the PROM21 chip, not to any of the CPU PCIe lanes.
This motherboard has a “flashback” feature, which appears to make it quite easy to load any BIOS, including modified ones. No external programmer required. I was able to modify the IFRs in the latest BIOS to insert a link to a hidden menu with PCIe options, including a “global hotplug enable” option, and enabling it appeared to do absolutely nothing.
ASRock has similar AM5 server-type motherboards that DO appear to have documented PCIe hotplug options in their BIOSes. I pulled the BIOS from their B650D4U3 board, and noticed a number of hotplug-related options in the AmdPbsSetupDxe module. It was about 3-4x larger by code size than the module of the same name in the BIOS for my board, so I tried replacing just that module using UEFITool, and that failed to POST.
I doubt it would be wise to try to load the complete BIOS from that server board, as the server board has a different topology, including a BMC.
I really don’t want to have to reboot the machine in order to switch out NVMe drives.
Any ideas what I should try next?