need help on recognize drive in PCIe card as data drive on Dell T5500 workstation (xeon e5645)

I installed fresh Windows 10 on a SATA SSD, then try to use NVMe drive in a PCIe card, it shows up in device manager under disk drives, but in disk manager, unable to initialize the disk either MBR or GPT, always getting "incorrect function" error, also tried:
1. pre-formated the NVMe drive when installed in a USB enclosure, still same error in disk manager
2. tried to use different PCIe card (16x or 4x) in different slots (PCIe 2.0 16x or 8x, no 3.0 available in T5500), same issue

Please help to get this work, much appreciated.

@wlee - Maybe the drive is formatted oddly? Are you sure the NVME is good? Was the drive new, or new to you (used)? Please confirm on some other non-OEM system, not in USB enclosure if possible

Try this, then once done, reboot back to Win10 and open disk manager, initialize it as GPT >> Diskpart >>
You can use diskpart in windows from OS (Direct from any CMD Prompt), or from installer ISO (press Shift + F10 on the first setup screen where you pick language)
Or from repair/troubleshoot options in the installer/setup. Remove all other drives, so you have only USB and target win10 drive. Then from CMD prompt do the following

1. Diskpart
2. List Disk << Here, identify what your target disk is by size/name etc, and make note of it’s #, you will use it’s # next - (If in windows, these #'s will match what you see the drives shown as in Disk Management) - Be sure you do not select your USB or main OS drive if in OS
3. select disk # << Here, instead of #, put target disk #, example >> select disk 0
4. clean
5. Exit

Reboot and then check and do as mentioned above in disk management.