Win 10 Standard NVMe vs Open Fabrics NVM OFA Reference Drivers

Hello everyone, thanks for giving me the opportunity to bring this subject into conversation.

In short, as you all should know, NMVe is a standard defined by a couple of companies. In their website, Open Fabrics, they mention some drivers like the inbox Win 10 and an open source one. They are listed here:
http://www.nvmexpress.org/windows-driver-microsoft-inbox/

And more detailed here:

http://www.nvmexpress.org/open-fabrics-alliance-nvm-express-window-driver-1-4-released-december-8-2014/

What bothers me is that they’re saying that the Inbox driver only supports NMVe 1.0e spec, and the open source one 1.2e. Since the SSD I have is an Intel 600p 256GB, just released, I believe it came already with this new spec, released in Dec 2014.

So, have any of you tried? Unfortunately I couldn’t install here, it game me an error, but I didn’t spend too much time on it though…

Thanks

Sorry, just found this post from an Intel employee asking for people to test it… I couldn’t find where to get the most recent…

http://lists.openfabrics.org/pipermail/nvmewin/2016-September/001395.html

https://svn.openfabrics.org/svnrepo/nvmewin/releases/

@hmaqueda :
Welcome at Win-RAID Forum and thank you very much for your interesting contributions and links.

Unfortunately I could not yet find a download link to the most recent NVMe driver, which has been developed by the related group.
Can you give me a direct link?

Thanks in advance!
Dieter (alias Fernando)

Sure, I found it here
https://svn.openfabrics.org/svnrepo/nvmewin/releases/revision_1.4/installations/64-bit/

You need to use the save-to method to get the files

Thanks for the link.
After having downloaded the related NVMe driver files I realized, that the related 64bit NVMe driver v1.4.0 is dated 12/08/2014 and not digitally signed.
These are the informations about the digital signature of the driver:

OFA NVMe driver digital signature.png



My comment:
Due to the fact, that the available NVMe drivers are nearly 2 years old and not correctly digitally signed, I will not try to get them installed.
Thanks nevertheless for your infos and the links!

Hi guys

Just to follow-up on this one, I’m still talking to them on their list. It turned out that, in order to get the recent driver, you have to compile it yourself from the source. Since I’m not good at it, they will release a binary 1.5 version soon… Also, I’ve confirmed that the Intel 600p is NVMe 1.2 compliant using AIDA64. Which means that, if the device/driver NVMe version matches, you’ll get much better performance than the inbox driver.

Here’s the thread if someone is interested…

http://lists.openfabrics.org/pipermail/nvmewin/2016-September/001407.html

Hey guys, just updating this thread here… They’ve just released the driver binary, it supports Win10 now and so on. I’m going to paste Ray’s email down here for reference:

Thanks a lot for feedbacks and tests. Anyone interested join their discussion.

@hmaqueda
I just tried to get the WDKTestCert NVMe driver v1.5 installed onto my Z170 system running the Samsung 950 Pro SSD as system drive.
Result:
The installation failed. This is what I got:

WDKTestCert NVMe driver.png

Hi,

I just wanted to tell you all I installed the OF ver 1.5 driver on my system and I didn’t notice any difference in speeds. Could be because I’m not running at full pci-e 3.0 link speeds and being bottlenecked by my motherboard.

If anyone else wants to try go to:
https://svn.openfabrics.org/svnrepo/nvmewin/releases/revision_1.5/installations/ and turn off driver signing. To do that you can try:

bcdedit.exe -set TESTSIGNING ON
bcdedit.exe -set loadoptions DDISABLE_INTEGRITY_CHECKS

And either press shift while clicking restart, and select ‘Disable driver enforcement’ at the troubleshooting page.
After that you right click the nvme setup file and click install. From there go into device manager and right click the microsoft nvme driver and select update and browse for it manually where you should see
the open source one. Note you might have to install a svn checkout utility to download those driver files. I used turtoiseSVN and right clicked the svn server path of the driver into that utility to download.

Happy testing…

Was able to fix the bottleneck issue after placing the card in the 2nd PCIE-X16 Slot as opposed to the X8 slot.

Did a couple of benchmarks with CrystalDiskMark:

With the Standard Microsoft Driver:


With the Open Fabrics Ver 1.5 Driver:

Anyone else interested in posting your benchmark results? Got to be some other curious minds out there?

As soon as I have the required time, I will install the brandnew v1703 of Win10 x64 onto different NVMe SSDs and do some benchmark tests using different NVMe drivers.
If I should be satisfied with the stability and performance of the OpenFabrics NVMe drivers v1.5.0.0, I will give them the Win-RAID CA digital signature.

Today I have done the first benchmark comparison tests.
Chipset: Intel Z170
Tested NVMe SSD: Samsung 950 Pro
OS: Fresh installed Win10 x64 Pro v1703 (Final "Creators Update")
Tested NVMe drivers:
1. Generic MS in-box NVMe driver v10.0.15063.0 named stornvme.sys
2. Generic OpenFabrics NVMe driver v1.5.0.0 named nvme.sys
3. Samsung NVMe driver v2.1.0.1611 named secnvme.sys


Here are the results:
1. Win10 in-box MS NVMe driver:

Z170_950-Pro_NVMe_MS-in-box.png



2. OpenFabrics NVMe driver:

Z170_950-Pro_NVMe_OFA-v1500.png



3. Samsung NVMe driver:

Z170_950-Pro_NVMe_Samsung-v2101611.png



My comments:

  1. As I already have written, the OpenFabrics drivers are not correctly digitally signed. They can only be used after having disabled the “Driver Signature Enforcement”.
    Interesting find:
    After having given the OpenFabrics driver a correct digital “Win-RAID CA” signature, I had no problem to get the driver installed, but I was not able to reboot into the OS. The OS tried to repair the boot procedure, but didn’t succeed. The only solution has been to choose the “Advanced Boot Options” and to disable the “Driver Signature Enforcement”. I have never realized such problem with any of the drivers, which had been digitally signed by me.
    According to the LOG file named SrtTrail.txt, which I found within the C:\Windows\System32\Logfiles\Srt subfolder, the driver named nvme.sys was monitored as being “corrupt” (although it was correctly digitally signed).
  2. It is wellknown, that the currently available Samsung NVMe drivers up to v2.1.0.1611 have a severe “unsafe shutdown” bug (look >here<).
  3. The generic MS Win10 in-box driver didn’t give me the best performance, but ran very stable.
  4. All 3 tested NVMe drivers gave my Z170 system a very good performances. The score differences were minimal and probably not even noticeable by the users while doing their daily work.

My conclusion:
As long as we don’t have properly working Samsung resp. digitally signed OpenFabrics NVMe drivers, I recommend to use the MS own generic Win10 in-box NMe driver.

Updated my PC recently:

Asus Z270G Gaming m-atx
Nvidia 1070GTX
Kaby Lake 7600K @ 4.9GHz
DDR4-3600 @ CL15
WD Black PCIe SSD M.2 (M.2 (PCI-e 3.0 x4)) @ 2.050/700MB/s

So I was first pretty shocked about the performance, but it ended up being a faulty bios setting and clean Win7 install which initially had bad performance.
I still wasn’t extremely pleased with the performance but i was ofcourse looking at ‘theoretical’ speeds.

Anyway, I’ve tried this driver and apparently it seems to work a bit faster for me.

Win7 Native MS Driver:


Opensource 1.5:



It’s actually quite a bigger jump than expected! :slight_smile:

Anyone know how to startup without having to press F8 all the time? I’m getting A CI.dll error or something otherwise. Im too lazy to find out myself xD

@MT1 :
Welcome at Win-RAID Forum and thanks for your contribution!

The OpenFabrics driver has yet not been correctly digitally signed.

Regards
Dieter (alias Fernando)

Ah gotcha. Thought it was.

Unfortunately now I keep getting the CI.DLL 0xc000428 "0xc0000428 Windows cannot verify the digital signature of …"
error unless i use F8 to disable driver enforcement.

Already deleted the two values with Bcdedit.exe and replaced the driver back to MS Native. Also Windows seemed to have lost its ‘activation’ status.

-.-

Edit: Fixed.

@Fernando ,

Thank you for running your benchmarks comparing the OpenFabrics Driver vs Microsoft and even vs Samsung driver. It was very interesting and deserves more study. You have always done an amazing job and I do appreciate the effort you have put into these benchmarks and the work you have done in the past.

Thank you.

PS: About running the OF driver and disabling driver enforcement I guess that is a personal choice the user would have to make. On my overclocked test machine I really don’t mind as I am pushing its limits in anycase. Thank you.

I do not agree with you regarding this point.
A storage driver, which has been released to the public, has to be correctly digitally signed by the manufacturer. This is the minimum what the user can and should expect.
If the OpenFabrics shouldn’t release a correctly digitally signed NVMe driver within the next 4 weeks, I will remove the related NVMe drivers from the start post of >this< thread. I don’t recommend to use a driver, which has been not even digitally signed.

You mean it could be dangerous and unsafe? If so I guess you have a point. Thank you.

Yes. As you can read >here<, I have not even been able to reboot into my OS after the successful installation of the OpenFabrics driver.