[Request/Guidance] Modify Bifurcation choices on Supermicro X9DRH-iTF

BIOS/UEFI Modding BIOS Modding Requests
1

Not sure if what I am attempting is possible or if I am even using the right approach:

X9DRH-iTF on latest BIOS (3.3) already has options enabled for bifurcation. HOWEVER, the strange/troubling thing is that the options for bifurcation only gives a single choice effectively meaning there are no options. For the physical x8 ports, only “x8” or “x8x8” choice is available, and for the physical x16 port, only “x16” is available. Opening BIOS in AMIBCP 4.55 shows exactly the same thing - only one choice available per port.

Here’s some of the output from ifrextractor on the “MiniSetupResourceSection” (BIOS appears to be Aptio IV). Note there is only one “OneOfOption” entry (this is for the x16 port):

0x2037B: 			EqIdVal QuestionId: 0x1DD, Value: 0x0 { 12 06 DD 01 00 00 }
0x20381: 			OneOf Prompt: "IOU3 - PCIe Port", Help: "Functions visible based on this setting : x4x4x4x4 (Fun 0/1/2/3 visible) x4x4x8 (Fun 0/2/3 visible) x8x4x4 (Fun 0/1/2 visible) x8x8 (Fun 0/2 visible) x16 (Fun 0 Visible)", QuestionFlags: 0x10, QuestionId: 0xDB, VarStoreId: 0x1, VarOffset: 0xE9, Flags: 0x10, Size: 8, Min: 0x0, Max: 0x0, Step: 0x0 { 05 A6 21 03 45 03 DB 00 01 00 E9 00 10 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 }
0x203A7: 				OneOfOption Option: "x16" Value: 4, Default, MfgDefault { 09 0E 4D 03 30 00 04 00 00 00 00 00 00 00 }
0x203B5: 			End  { 29 02 }
0x203B7: 		End  { 29 02 }
0x203B9: 		SuppressIf  { 0A 82 }
0x203BB: 			EqIdVal QuestionId: 0x1DD, Value: 0x0 { 12 86 DD 01 00 00 }
0x203C1: 				EqIdVal QuestionId: 0xDB, Value: 0x4 { 12 06 DB 00 04 00 }
0x203C7: 				Not  { 17 02 }
0x203C9: 				Or  { 16 02 }
0x203CB: 			End  { 29 02 }

Every bifurcation option listed on this motherboard is the same - only one “OneOfOption” for each entry, with the difference being which one option is presented. I have also tried different combinations of BIOS options and PCIe cards - the available BIOS options do not appear to change.

Looking at other similar gen Supermicro motherboard (X9DRi) in AMIBCP shows what I would expect - multiple choices (i.e x4x4 or x8) per port.

So I suppose what I am asking is: is there a way to add/enable these additional choices? I can’t seem to figure out how I would. Have played with the various UEFITools/UEFI Editor as well as different versions of AMIBCP.

Any guidance would be appreciated. Thanks!

1 Like
2

UPDATE: Decided to use datasone’s modded GRUB shell to test changing the NVRAM variables manually and test the result. No sense in hacking up the BIOS if it wouldn’t work in the first place (I think, anyway).

I was fairly confident I would at least be able to get the x16 port to work. I also had high hopes for x8 slots 2, 3, 6 and 7 as they didn’t share lanes with any onboard devices.

TL;DR: No dice, weird results. Questioning sanity as Insomnia continues…

X9DRH PCIe Routing

x79drh-pcie-lanes
x79drh-pcie-lanes795×431 58.1 KB

I made some assumptions when doing this testing, first that the actual NVRAM configuration values were enumerated in the same order as the items were listed in the setup text.

BIOS Setup text:
Functions visible based on this setting : x4x4x4x4 (Fun 0/1/2/3 visible) x4x4x8 (Fun 0/2/3 visible) x8x4x4 (Fun 0/1/2 visible) x8x8 (Fun 0/2 visible) x16 (Fun 0 Visible)

Therefore, hopefully:

x4x4x4x4 = 0x0
x4x4x8 = 0x1
x8x4x4 = 0x2
x8x8 = 0x3
x16 = 0x4

This order was consistent with the NVRAM values for the BIOS defaults for said slots (i.e the x16 slot default was 0x4 and the paired x8 slots were 0x3) so I am pretty confident about this. Also compared other Supermicro X9xxx BIOS with working options and it used the same mapping.

Testing:

I used the IFRExtractor tool on the “MiniSetupResourceSection” to get the values used for testing. After installing cards, I would boot to the modified shell and use “setup_var_cv” to set NVRAM values. Several times I power cycled/rebooted and re-read the values to make sure they were not being overwritted/reconfigured and this did not ever seem to be an issue.

I tested with two different noname e-bay 8x bifurcation cards, a dual M.2 and dual U.2, not that it should really matter. Test OS was Fedora 39, kernel 6.6.latest…

IIOU(CPU) 1 IOU 2 - PHY SLOT 2 (x8) and 3 (x8)

  • Default x8x8 (0x3), offset 0xA3, size 0x1
  • Installed both PCIe cards (slot 2 and 3)
    – Test @ default (0x3): Sees first drive in each slot (as expected)
    – Test @ x4x4x4x4 (0x0): Only sees first drive in slot 3
    – Test @ x4x4x8 (0x1): Sees first drive in slot 2 and slot 3
    – Test @ x8x4x4 (0x2): kernel panic?!

IIOU(CPU) 2 IOU 2 - PHY SLOT 6 (x8) and 7 (x8)

  • Default x8x8 (0x3), offset 0xE7, size 0x1
  • Installed both PCIe cards (slot 6 and 7)
    – Test @ default (0x3): Only sees first device in slot 7 (???)
    – Test @ x4x4x4x4 (0x0): Only sees first drive in each slot
    – Test @ x4x4x8 (0x1): Only sees first drive in each slot

IIOU(CPU) 2 IOU 3 - PHY SLOT 4 (x16)

  • Default x16 (0x4), offset 0xE9, size 0x1
  • Installed x8 to x4x4 u.2 card
    – Test @ default (0x4): no devices (???)
  • Sanity check: Installed x16 VGA card, verified working and x16/8GT link via lspci -vvv
  • reinstalled x8 to x4x4 u.2 card
    – Test @ x4x4x4x4 (0x0): no devices
    – Test @ x4x4x8 (0x1): no devices, reboot hangs at POST 0x91
    – Test @ x8x4x4 (0x2): no devices, reboot hangs at POST 0x91
    – Test @ x8x8 (0x3): no devices
  • Installed x8 to x4x4 m.2 card
    – Test @ x8x8 (0x3): no devices
    – Test @ x4x4x4x4 (0x0): no devices

I did some additional testing on CPU2 IOU1 (Slot 5) not listed but had no progress (always saw first device despite config).

I’m really at a loss as to what is going on - especially with the x16 slot. Unfortunately I only know enough to be moderately dangerous here. I don’t know if there are weird compatibility issues with my SSDs or my cheap aftermarket PCIe cards, or what else could be going on. All of my SSDs are Intel DC series (I even tested with one of those $10 16GB Optane m.2 drives).

If I get another burr up my butt and do additional testing or have a breakthrough I will update this thread, otherwise I will let it die.

New to the board so Mods, please be gentle if I am doing something dumb here. Thanks!

1 Like
3

Personally, I hope you’ll come back to this issue.
I decided to go the brute force route. I searched for IOU3 and found “setup” and pulled out the “MiniSetupToolsSection” module from it.
HexEditor changed the setting for x16 “0x04”. I replaced the bits that changed the setting in “X9DR3P9.B20” in the BIOS accordingly (not stupidly).
I flashed the module back and wrote it into the BIOS. As a result - a brick, although it does start.
I restored it via the network (BMC).
The whole problem is that the lack of bifurcation can be bypassed with a plx bridge, the cost (at the moment) is slightly higher than the motherboard. That’s the main problem.