Intel® Rapid Storage Technology (RST) remapping of PCIe storage devices (HP)
https://h20195.www2.hp.com/v2/GetDocumen…me=4AA7-8270ENW
’Remapping’ seems to be the explanation for some ‘disappearing’ NVMe devices when having RST controller configured as ‘RAID’ in bios.
Seems that rst and rste do use different modes for creating NVMe RAID- RST seems to remap the drives (even if on PCIe lanes connected directly to the CPU) to the RAID controller of the chipset/ PCH while later RSTe versions (VROC) rely on certain (Xeon?) CPU capabilities for handling NVMe RAID?
C610/X99 chipset/ PCH can- like newer chipsets- be configured to present either as 2822 or 2826 and RST RAID capabilities can be configured/ restricted via bios- link. But that’s dependant on PCH device ID, 8D40h can be changed, 8D44h (C612) seems to be fixed on RSTe, 8D47h (X99) seems to be fixed on RST.
Found a spec for RAID layout on disk. RAID DDF Spec 2.0. It’s pretty technical, but not tooooo bad…
I would think that the above layout would also be applied to NVMe drives?!?!?
So, aside from PCIe remapping (I didn’t know that was a “thing”), what other differences/“gotcha’s” do/might exist with a software-RAID implementation on top of NVMe compared to good-ol’-fashioned spinny disks or SATA ssds?
Found a YouTube video with some hints on x4 NVMe RAIDs, apparently, there is a bottleneck limiting the read speed to 8GiB/s:
https://www.youtube.com/watch?v=lzzavO5a4OQ&t=5s
AMD Boards might support x4 NVMe Raid better than Intel ones.
MSI b450 and z370 Boards support NVMe Raids in m.2. Slots, but a second m.2. Slot ith x4 Lanes is rare. Gigabyte Support tells also, CSM must be disabled, but only AORUS ultra and above support bifurcation. Biostar Boards seem not to support Bifurcation, but NVMe RAID also from B550 if two m.2. ports are available.
ASUS has a very good FAQ-Page on Bifurcation Support:
https://www.asus.com/support/FAQ/1037507
ASUS Support shall recommend X399 and X570 and z Chipsets for NVMe RAIDS, but actually all ASUS Boards as listed in this FAQ all support ASUS Hyper Raid Controller and similar in the first PCIe Port.
ASROck supports NVMe RAID in Bios X399 / X570: Advanced -> AMD PBS and set “NVMe RAID mode” to <Enabled >, CSM must be disabled.
Interesting. When I picked up a used 4x M.2 slot PCIE adapter card, it didn’t even occur to me that I might have better perf using 2 separate 2 slot cards on my dual xeon mobo (same kinda mobo pcie lane allocation between procs as the psuedo-dual-proc AMD CPU).
That said, finding a dual slot that is actually a dual M-key and not a hacked up M-key plus B-key sata is surprisingly challenging… It is bizarre how many “single M plus single B” variations there are and so few true dual M.2 NVMe compatible adapter cards. One of the only dual slot adapters I’ve found that I might buy at some point is the supermicro AOC-SLG3-2M2. But since it costs around $50 which isn’t bad for a Supermicro name-brand card, and I can get 4x slot adapters around $75ish, or a single slot adapter for $6… Well, so far I figure either pay $6 for a cheapie adapter OR get a 4 slot ASUS card. I have both and haven’t had any problems.
I’m still hoping someone will release an open source EFI driver for at least read-only RAID support for bootable NVMe arrays. The driver would not need to create or maintain arrays, but be barely functional enough to scan and read the RAID metadata from disk and understand the data layout barely enough to get the kernels loaded and leave it to the OS drivers to handle the rest.
You can search for Motherboards with multiple m.2. slots here:
https://geizhals.de/?cat=mbxeon&xf=11924_2%7E8405_GEN34X4
Geizhals has the best metadata of all portals. You just have to find a mobo and look into its manual afterwards in order to find out whether it supports NVMe Raid. In brief, ASUS does best support ASUS Hyper RAIDS, Gigabyte does not offer Bifurcation but if the mobos have multiple m.2. ports then they can also be in a NVMe RAID, Biostar and MSI as well. ASRock offers this feature only with X399 and X570 Chipsets.
According to my knowledge a bootable NVMe RAID array doesn’t require on-board M.2 slots, but the BIOS must support the "Remapping" of the used PCIe slots.
Well, I tried out lots of Mainboards now. AMD Chipsets will most likely support NVMe RAID with the AMD Driver from the Homepage if there are at least two m2 slots on board: AMD RAID Driver (SATA, NVMe RAID)
https://www.amd.com/de/support/chipsets/amd-socket-am4/b450
You can use any port, either m.2. directly connected to the CPU or connected to the Chipset via an adapter PCIex4 Adapter Card. I got problems with the ASUS Hyper Card though. It did not work well with the AMD RAID configuration file in the driver, so I could not install windows. I still use an ASUS Mainboard, one m2 in the Slot another in the Adapter Card.
The info on m2 slots was from the Gigabyte support. They say if there are multiple on-board M.2 slots on an Intel Mainboard, then you can build a RAID. Indeed I could using the Gigabyte AORUS B550 Elite AX V2. Anybody wants it? I switched to ASUS now. I guess Gigabyte BIOS supplies the PCIe “Remapping” only for those boards having multiple m2 slots. If there is only one slot, then I could not add the NVME to an Array on an Intel Board. Anyway, AMD NVMe is functional now, only Intel Chipsets make problems because they want to promote Optane Technology. I think you can even combine SATA SSD in an m.2. Slot with NVMe SSD which actually does not make sense, but it seems to be possible. And ASUS does not support NVMe RAID officially, except if you use the Windows install driver from the AMD Homepage. It seems you can build Raids with NVMe and SATA combined when in the m.2. slot. But once you have built an NVMe RAID you cannot use the second m.2 slot for SATA nor the SATA ports 5&6. You have to connect the Sata SSD with a cable to a SATA port then. You can neither build a SATA RAID additionally to NVMe RAIDs. If you want SATA RAID additionally for Data you have to switch the Sata Controller to AHCI and configure a software RAID with “Dynamic Stripped Volumes” within Windows. Performance is great also, even better for small files.
Benchmarks:
NVMe RAID on B450m Mainboard
SATA RAID 2x 128GB Samsung CM871a
Single 5TB HDD WD50EFRX
Now we just have to find out what makes the difference in the BIOS. TUF B450m Plus seems not to be RAID capable, but TUF B450m PRO seems. Same is valid for Gigabyte z370 hd3 vs. z370 hd3p.