[Problem] NVMe Support for TYAN S5512WGM2NR running TrueNAS Scale

Hi,
I followed the guide on this forum for adding NVME support to my bios (AMI Aptio IV).

Firstly, my BIOS was reporting as version 8.B14 on the machine. I used AFUDOS with the /O option to dump my BIOS prior to flashing.
Secondly, the BIOS on the manufacturer’s site (TYAN S5512WGM2NR) is version 2.05.
I tried modifying the backup BIOS that was on my machine in MMTool 4.50.0.23 with NvmExpressDxe_Small in compressed format - the standard NvmExpressDxe_4 gave a too large error.
All was added ok, and I could see the entry in the table for NVME. I saved the image, and flashed it onto my machine. All I got from that point onward was a boot loop

As I couldn’t stuff up my machine any more, I decided to take the BIOS that was on the manufacturers site, and apply the same mod to that. Flashed it, and the boot loop stopped!
I then proceeded to add the NVME devices to my system. I am using a QNAP QM2-2P10G1TA to add NVME support.

With one of the NVME drives installed (I am using Western Digital SN550 500GB) I am able to get through POST, and the drive shows in my OS (TrueNAS SCALE).
With a second drive installed, the BIOS does not detect any bootable drive whatsoever, and boots to an EFI shell.

I should add, that I am not trying to use the NVME drives for boot, they will be used for a ZFS cache.
I should also add, that my BIOS backup sits at 4MB whereas the BIOS downloadable from the manufacturer is 8MB - The BIOS at boot shows that it has 8MB capacity.

Question 1. Is there a way I can add the existing functionality to my original, more up-to-date BIOS?
Question 2. Why can I only use a single NVME drive, and can this be fixed?

Any help will be gratefully received!

Edit1 by Fernando: Thread moved into the appropriate Sub-Forum and unneeded blank lines removed (to save space)
Edit2 by Fernando: Thread title specified and customized

Funny idea to use a card with a PCIe switch that offers PCIe 2 x 2 to an NVMe disk, and is switching a third device (ethernet) on the same lanes. This way there won’t be much of a speed difference legt compared to a SATA disk. (Info taken here https://www.qnap.com/en/product/qm2-2p10g1ta/specs/hardware )

Regarding the other questions: Post links to/ attach the files you used and created.

@danielhitch : Welcome to the Win-RAID Forum!

If you don’t want to boot off an NVMe SSD, you don’t need to add an NVMe EFI module into the BIOS.
Here are my answers to your questions:
1. It is possible to get NVMe boot support by inserting the NVMe module into the dumped BIOS Region.
2. I suspect, that the PCIe slot doesn’t offer enough data transfer lanes.

My questions:
1. Which OS are you running?
2. Why don’t you want to install the OS onto an NVMe SSD?
3. How many PCIe lanes are available for your QNAP QM2-2P10G1TA PCIe card?

Good luck!
Dieter (alias Fernando)

Hi All,
Thanks for the responses!

@lfb6 - I am aware of this. The motherboard is installed in a space constrained 1U storage shelf. I only have access to 1 full-size PCIe slot which, in-turn is available via an ‘L’ shaped riser. This also constrains the length of card I can put in the enclosure so, once you take into account what I would like my system to do, and what space is available, there was really only the one choice.
I will attach links once I have got over the 3 post limit.
I am aware of the bandwith limitations of the device however, as it is switched, I should be able to get a reasonable throughput dependant on the demand of the 3 resources available on the card. i.e. if I am only using the 1 NVME device, I would be getting a reasonable throughput. If I am using all 3 devices at once (2 NVME and 1 10GBE LAN) then yes, performance won’t be that great!

@Fernando - I am using TrueNAS Scale (as mentioned in the above post)
It isn’t a case of not wanting to install the OS onto the NVME, it is that I already have the OS on a separate SSD and would prefer to keep my setup as a 1 drive per function / purpose.
The device has a switched PCIe Gen2 x4 interface with an oversubscription of 1.5:1, in reality, each device on the card has a maximum throughput of PCIe Gen2 x2
Without the NVME support added to the BIOS, the machine refused to boot (would just boot straight in to the BIOS setup menu)

@danielhitch Would that slot be slot#5 or slot #6? This is a rather old board, there may be some problems dealing with that switch. You did check the PCIe settings in bios- "Detect Non-Compliance Device" looks interesting for example (but should be disabled as default).

What cpu is installed?

Bios backup is possibly done with afu… tool, will just backup the bios region (4MB), not flash descriptor, GbE, ME (SPS2)- the other/first 4 MB. Use UEFIToolNE to see structure.

@lfb6 - The card is in slot #6. The CPU is a Xeon E3-1265L. I am guessing my BIOS backup is useless then if it is missing parts! D’oh!

Absolutely not! That’s where (most) of board specific data normally are stored! MAC address may in GbE, you can try to dump with Intel ME tools. FD and ME are part of the bios and can be taken therefrom for creating a complete firmware image with your own bios region…

Xeon E3-1265L has to be Ivy Bridge (E3-1265L V2) since SB didn’t have a E3-1265L, only an E3-1260L… And if it slot #6 then I don’t understand your plans, seems that this is the slot connected to the CPU physical PCIe 16, but with 8 lanes connected and since it’s Ivy Bridge CPU this should be PCIe3. Why did you go for a card with a PCIe2 switch that uses just 4 lanes and keeps you in trouble- possibly because of that switch?

EDIT: Another thing might be the configuration of the pcie- slots. A simple adapter card with 2 NVMe slots would possibly work on a PCIe slot configured 4 lanes x 2 but not automatically on a slot configured 8 lanes x 1?

@lfb6 It looks like you are correct with what has been backed up. I have also reached out to TYAN to see if they can provide me with their latest BIOS given that the one on their site is vastly out of date in comparison to the one on my machine. I won’t hold my breath though, as the board is majorly out of support!

My processor is a Sandy Bridge based processor (just E3-1265L) according to Wikipedia, this is an OEM only offering. I could possibly purchase an Ivy Bridge based processor if this would be better compatible…?

As mentioned previously, I am rather space constrained in the enclosure. It is a 1U height storage shelf and I require both 10GBe and a fast(ish) cache. There are two cards that offer this combined functionality - The QNAP that I have purchased, and the Synology E10M20-T1 (which is a PCIe Gen3 device). I chose the QNAP because the Synology would be too deep to fit into the enclosure (the QNAP only just fits). I did not know at the time, that the addition on the PCI switch would cause so much issue.

@danielhitch Normally if everything is made according to specs this card should work. The switch should be transparent to the system, and 4 lanes in a x16 slot shouldn’t be a problem. There’s one very rare old thing that I did experience here again regarding a passive ASUS NVMe x4 adapter:
Incompatibility between Mainboard and M.2>PCIe Adapter Card? which goes back to a rather old story:
http://yannickdekoeijer.blogspot.com/201…controller.html

I don’t think this will help you, but maybe it’s worth a try? As written before: For what you’re planning to do there’s no bios support for NVMe disks necessary, the card itself doesn’t need to be recognized by the bios…

@lfb6 I am just trying to get my machine to post, and availability to be listed with both NVME slots populated at present.

As the machine doesn’t post with both slots populated, I am still leaning towards a BIOS issue - as you said, it should just work.

I’ll check out the post you linked above.

Thanks!