I recently made some necessary updates to our lab by upgrading some of our older servers to handle storage.

I currently have 3 poweredge R720's on my rack and I wanted to use them specifically for Ceph storage handling.

I have installed the GY1TD card which has a PEX 8734 switch internally and can handle x4x4x4x4 bifurcation. I had also replaced the sas backplane with the necessary one to allow u.2 drives to work. All these parts are Dell parts and the drives light up and looks like they connect.

The problem is the following..

If I have the drives connected at boot, the boot process gets stuck at "initializing firmware".

If I remove the drives out of the caddy but I have the backplane and pic card connected then the server boots fine. But if I put the drives back in then the drive caddy lights up green and looks like it's doing something but I can't see the drive at all on the host. fdisk, blkid, lsblk nothing shows the drives.

I do not want to boot from these drives but I do want to use them strictly for storage on ceph as the poweredge servers have all been updated to 100Gb fiber links in-between the cluster.

I have also removed the perc card that was in the servers originally.

What can I do to make this card work ? I want to create an all flash ceph cluster and im having a real hard time with it.

lspci output below

04:00.0 Ethernet controller [0200]: Mellanox Technologies MT27520 Family [ConnectX-3 Pro] [15b3:1007] `Subsystem: Mellanox Technologies MT27520 Family [ConnectX-3 Pro] [15b3:0007]` `Kernel driver in use: mlx4_core` `Kernel modules: mlx4_core` 05:00.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch [10b5:8734] (rev ab) `Subsystem: Dell PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch [1028:1f84]` `Kernel driver in use: pcieport` 06:04.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch [10b5:8734] (rev ab) `Subsystem: Dell PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch [1028:1f84]` `Kernel driver in use: pcieport` 06:05.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch [10b5:8734] (rev ab) `Subsystem: Dell PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch [1028:1f84]` `Kernel driver in use: pcieport` 06:06.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch [10b5:8734] (rev ab) `Subsystem: Dell PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch [1028:1f84]` `Kernel driver in use: pcieport` 06:07.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch [10b5:8734] (rev ab) `Subsystem: Dell PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch [1028:1f84]` `Kernel driver in use: pcieport` 0d:00.0 PCI bridge [0604]: Renesas Technology Corp. SH7757 PCIe Switch [PS] [1912:0013] `Subsystem: Renesas Technology Corp. SH7757 PCIe Switch [PS] [1912:0013]` `Kernel driver in use: pcieport` 0e:00.0 PCI bridge [0604]: Renesas Technology Corp. SH7757 PCIe Switch [PS] [1912:0013] `Subsystem: Renesas Technology Corp. SH7757 PCIe Switch [PS] [1912:0013]` `Kernel driver in use: pcieport` 0e:01.0 PCI bridge [0604]: Renesas Technology Corp. SH7757 PCIe Switch [PS] [1912:0013] `Subsystem: Renesas Technology Corp. SH7757 PCIe Switch [PS] [1912:0013]` `Kernel driver in use: pcieport` 0f:00.0 PCI bridge [0604]: Renesas Technology Corp. SH7757 PCIe-PCI Bridge [PPB] [1912:0012] `Subsystem: Renesas Technology Corp. SH7757 PCIe-PCI Bridge [PPB] [1912:0012]` 10:00.0 VGA compatible controller [0300]: Matrox Electronics Systems Ltd. G200eR2 [102b:0534] `DeviceName: Embedded Video` `Subsystem: Dell G200eR2 [1028:048c]` `Kernel driver in use: mgag200` `Kernel modules: mgag200`