Ah. That explains that. Tinyupload isn’t working across 3 pcs and a handful of browsers.
This will nix the problem for now: https://drive.google.com/file/d/1STzy1Z0…iew?usp=sharing
@xurious - Yes, tinyupload has been failing on uploads for a few days now, this happens sometimes 
Should you be able to boot server to BIOS? I assume so, but I’ve never used any server, so not sure how they work. Yes, it would be great if you have some other retail CPU to try here, any with CPUID 50654-50657 should work
Do these boards have a debug LED onboard? If yes, what codes do you see, and what does it stop on? Have you tested with single CPU and one stick of memory, have you tested different sets of memory?
If they do not have debug LED, do you have a PCI debug POST card, if these have a PCI slot to use one in?
It would be great if we at least knew these CPU’d CPUID, but at least we know the mod BIOS covers all other known ES, so these should boot too based on all the others being OK, they could possibly be faulty unless they do need some old microcode for some reason.
I will make you SAGE BIOS with the same microcodes as this server tonight 
I assume the server is posting, but there is no onboard video and the dedicated pcie gpu does not present video.
There is uart debugging headers, but I don’t have the tooling/skills for that.
The ideal way to access it would be through the onboard BMC. At present I can not get it to request an IP address and don’t know the default or current ip.
The only onboard debug led just stays red at the moment. I’ve done the standard of ensuring all items are fully seated. I also don’t have a debug pcie card, may be time to finally acquire one.
I know these servers ran windows hyperV with the standard windows sever GUI, I can take the main ssd and boot it in another server.
One thing I have not tried is adding a standard pcie nic, booting a debian live image, and seeing if it pulls dhcp from there.
I don’t have any retail chips, but I will try with the other ES ones.
@xurious - You can’t assume it’s booting past POST 
Especially if you can’t get any visual output << Did you try a few PCIE cards, removing the original? Or can you not use standard PCIE Card, I can’t tell from the single image at the link above.
Can you ping it, or see if from a router? Debug LED = Red? So it’s not alphanumeric?
Yes, if there is PCI slot, get a $3 PCI Debug post card, it will let us know if stalling or posting etc.
So, you can’t get into it’s BIOS?
@Lost_N_BIOS Hi again, finally had the chance to flash your latest mod bios. But I’m getting stuck on code 66 with 1 long and 3 short beeps with it. QL1K ES CPU Platinum 8160 <- with those cpus.
Gonna flash back to the old bios and see if it goes back to rock solid again.
EDIT: Ok, after flashing back to the old bios (working fine) and then flashing back to the new one again it started working as supposed. Might have been some extra hw that I removed between the first flash and the second that did the trick.
So it seems to be all good! Sweet work as usual! 
Thanks alot!
@Babar - Not 100% sure what 66/bb error and those beeps mean, but maybe this is a memory or settings issue, did you try to clear CMOS?
QL1K confirmed working with latest 6201 mod BIOS already by @levelud here at post #407-409 levelud what is your CPU models, same 8160’s?
Yes, confirmed on page 2-3, error 66 1 long and two short beeps (no one mentions 3 short beeps, so assume it’s two?) is a memory related error (no memory specifically). May also be PXE Init, but still memory related.
I see one long and three short is VGA issue, in the SAGE manual. Did you test with single socket CPU and one stick of memory? Did you try other memory, just as testing?
Ahh, just edited my post before refreshing to see your reply… Anyway, all good now! 
And yeah, it might have been a vga issue, because I’m running 3 gpus off the first pcie slot using bifurcation, and I removed those before the second try.
@Babar - Great to see you got it going!! 3 GPU’s via bifurcation, wow!
Nice it’s working for you now 
Jupp, the plan is more gpus, just need the cash
Maybe now that ppl are dumping gpus for the new 3000 series nvidias…
Folks, I want to start by saying a big thanks to everyone who started and is maintaining this project. I specifically built my system around the C621e because of reading this thread and realizing the value of being able to use B0 stepped chips. Glad to say I finally built the system today, flashed the BIOS and was able to boot without errors. You’ve created an incredible source of value for the community, thank you!
--------------
Quick newbie follow-up that I couldn’t find answer to in the thread: I can’t seem to get the multiplier on my 2 QL1K’s to go above 22 (on the 10-32x scale). Am I missing something? I set the BIOS to both “Core Optimizer” and “Engine Boost” and while it does max the CPU to a constant 22x, it still won’t go beyond that point.
@Morockin - Disable Xeon Turbo Charger, I think I think I read this messes with multiplier, not in a good way. Also, disable Engine Boost may help too. Core Optimizer sounds like what you want, provided turbo is enabled.
Looking at the specs for QL1K, stock multi is x18, are you sure max would be x32? Maybe @babar or @levelud can help you more about these CPU’s and getting CPU multi setup correctly.
I assume you disabled C1E and EIST already, you may need to leave C3 and or C6 enabled for turbo to work properly on these models, not sure. Did you try manually setting higher than 22 for the core ratio, and set 32 or so at Core Max OC Ratio?
I do see this though, so you may be correct, but for 32 always you’d need some kind of other BIOS mod I assume (max turbo core kind of mod)
Thank you for taking the time to respond. You’ve reminded me of a few of the OC variables that I haven’t played with yet, so I am going to methodically work through the combinations to see what I can “unlock”…
@Morockin - You’re welcome! If you don’t make any progress, I’m sure someone will stop in here soon and tell you how to get the higher CPU multipliers working, should be same/similar for all these CPU’s
So, after 3-4 hours of variable mixing, I’ve been able to squeeze out about 20% greater performance with OC tweaks but still can’t get back the 22x multiplier. If anyone has any suggestions, please let me know!
Actually I haven’t touched the multi or any kind of overclocking on this board. Strange to say, because thats usually the first thing I try. But for this I’m just looking for total stability. I’m already amazed at how fast it runs everything I throw at it, even in a VM…
Good news, I was able to get multiple cores unlocked to the full 2.8-3.2ghz with a few tweaks to the CPU C/P states and was able to preserve most of the energy savings/idling to ensure that I wasn’t running hot all the time. Thankfully none of it required playing with the BLCK settings. Screenshot attached.
@Morockin - Great to see you were able to figure out the multi’s and turbo! So, looks like under full load, one core can hit 3.3, another can do 2.9, and then rest limited to 2.2
That sounds about right for modern Intel Turbo without some kind of Turbo mod, which I think you can only do on X99
So which C state did you need to leave enabled, and which could be disabled? I assume C3 and or C6 had to be left enabled, right?
@Morockin Thats sweet! If you have the time could you give a quick walkthrough of what needed to be changed to unlock the full potential? Maybe I’ll acutally have to try it, for the hell of it
Hello,
I am using a Asus WS-C621E-SAGE, and I am using two intel non ARK Xeon CPU’s, which are label as P-8136, which is close to an ARK Xeon of 8176, with slight lower base and turbo frequencies.
I was running version 6003 with no issue, and I attempted to update to version 6201, and now the system will not post, because I do not think it is detecting the CPU’s correctly.
Could you please provide version 6201 with P-8136 cpu support?
Please let me know if you need any additional information from me? Or if you would like me to start a new thread as well?
Best Regards,
Al
---------------------
Full disclosure, I purchased these CPUs from an ebay seller located here:
https://www.ebay.com/itm/352653551012
I have tried to reach him to provide an updated mod bios for the WS-C621E-SAGE board; however, he has not responded to my request.
If it helps, here is additional information about the CPU’s from the details of the ebay page:
Intel Xeon Platinum P-8136 OEM CPU LGA-3647 2.0GHz 28-Core 165W SR2YN ~ 8176
Item specifics
Condition: Seller refurbished :
Seller Notes: “Very good condition. May have minor scratches on the thermal lid and/or PCB. Please read description for system compatibility.”
All-Core Non-AVX Turbo Frequency: 2.7 GHz Service & Support: Limited 180 Day Warranty
Max Turbo Frequency: 3.5 GHz Clock Speed: 2.0 GHz
Socket Type: LGA 3647-0/Socket P0 L3 Cache: 38.5 MB
Processor Model: Intel Xeon Platinum P-8136 MPN:
SR2YN
Number of Cores: 28 Processor Type: Xeon Scalable Processor
Thermal Design Power: 165 W Brand: Intel
Stepping: B1 UPC: Does not apply
genuine Intel Xeon Platinum P-8136 CPU SR2YN. This data center customized OEM (not ES/QS engineering sample) processor has very similar specifications to the retail Xeon Platinum 8176 (SR37A), except for a maximum supported memory speed of 2400MHz.
This processor requires a BIOS microcode of B1 production stepping.
Compatible Products:
Asus: WS C621E SAGE, Pro WS C621-64L SAGE/10G, ROG Dominus Extreme motherboards (modded BIOS with a microcode patch required***)
*** Modded BIOS can be flashed via USB BIOS Flashback without the need of a natively supported processor.
Thanks in advance for any assistance you can provide,
Al
@ThatGuy2020 - Try what Barbar had to do here, one page back at post #445 - Allow B0 stepping on Asus W621E Sage (30)
If that does not help, while you are back on working 6003, check with HWINFO64 and or in BIOS main/info area, and see what CPUID this CPU is using.
You do not need to start a new thread. If you can’t get it working, then I will take microcode being used from 6003 (link me) and put in 6201 for you