I've posted this in r/homelab and r/HomeNetworking and have only gotten two recommendations which were functionally the same (Mellanox SX6036 and SX6012; IDK how to enable what's necessary on these), perhaps yall have answers.

I'm looking to eventually deploy RoCEv2 in my home lab but am not 100% sure on which switches I've seen can support it nor which have noob friendly interfaces (i have very little switch UI exposure). I know ECN, PFC, DCBx, and ETS are the required features, but I've read you can get away with the former two. Do you need all 4 or can just the 2 get you what you need?

For switches, I've found a small selection. Am I correct in my analysis' on them?

Arista DCS-7050QX-32S: p. 4 under "Quality of Service (QoS) Features" it lists all 4. This will work

Brocade BR-VDX6940-36Q-AC: p8. under "DCB features" lists PFC, ETS, DCBx by name and I think "Manual config of lossless queues" would be the other. This may work

Edge-corE AS77[12,16]-32X: I thought that I read NOS (or whatever OS this thing uses) has the 4 things I need. This may work

Dell S6010-ON: the last bullet on p.1 says "ROCE is also supported on S6010", but is that v2 or not? I see PFC, ETS, and "Flow Control", so I'm not 100%

Cisco Nexus N3K-C3132Q-XL: this has ECN and PFC but none of the other 2 features by name. This may work

I would get at least CX3's for this as they're the cheapest and meaningfully utilizing 50/100G is a long ways off for me. The goal of this would be to enhance my planned storage (a pair of ? nodes hooked into at least one DDN shelf running BeeGFS w/ ZFS backing) and compute (multiple Dell C6300/Precision 7820 type machines running suites like QuantumESPRESSO) systems

I bought a couple of Dell R720xd servers a while back. One for Proxmox and one for TrueNAS. They work great for my needs and I’d like to upgrade them for some basic local LLM and other GPU workloads.

I’ve seen a number of folks post on YouTube with working Tesla P40s in their 720xd servers. So I buy a couple along with the wiring one of the posters linked.

I also picked up 1100W PSUs and threw those in there. iDRAC and the BIOS are updated to latest.

However, when I try to boot with the GPU installed the server won’t boot, the PSU blinks orange, and there are zero logs in iDRAC as to what the issue might be. This happens even on a dedicated 20A circuit with no other load.

Anyone out there have any ideas?

ETA I got them working. I’d tried two different cables and neither worked for me, but this cable from Amazon did: GinTai 8(pin) to 8(pin) Power Cable Replacement for DELL R730 and Nvidia K80/M40/M60/P40/P100 PCIE GPU

Hey everyone, I asked this on r/homelab a few days ago but didn't get much, so I'll ask:

What are some homelab projects for someone who genuinely couldn't care less about self-hosted software. I use the software I use and have no real need to branch out, but I love messing with used enterprise hardware. I currently have a few used 13th gen Dell PowerEdge servers with more on the way, so I'm looking for some cool projects where the hardware matters significantly more than just running *arr stacks or Plex. Here are what I'm currently looking to try out:

• Proxmox HA w/ Ceph
• NAS w/ JBOD extensions
• SAN w/ attached ThinOS hosts or PXE boot server
• Multiple CAD workstations in one server
• Tape backups
• Multi-node servers
• Ludicrous network designs/speeds
• Odd enterprise server builds

So what am I missing here? What are some cool hardware-oriented projects to try out? Thanks in advance!

Can anyone help me set up BitRaser wiping appliance to wipe in-system and in-array drives, I'm in Rochester, NY!

I have depoyed 12 Dell C6420 (Dual Xeon 8255c - 165w, 512GB RAM) and 4 Dell C6525 (dual EPYC 7502 and 512GB RAM). All of them currently have a RAID/Riser card (BOSS S1) for boot. They are all diskless servers, with a dual 25GbE NIC and dual FC 16Gbps HBA. Disks are presented from a NetApp A700s with about 500TB effective capacity.

As every Raid Card + a M2 drive for boot ESXi, It would consume about 200-300w based on my estimated. I wonder should I switch to SAN boot to save a little bit of power, and it's also simplify the infrastructure as less components then lower failure rate.

The reason behind is that i purchased 1 rack, they are limited 7KW/Power Grid and I dont want the 2nd rack just for power.

I’m currently running two Ubuntu servers on my PCs, but my storage needs have outgrown this setup (I store/access a lot of multimedia content for video editing/production). I’m considering migrating to a dedicated storage server, but I’m not sure where to start or what exactly I need for this transition.

At first I was planning on building a new PC and throwing external hard drives as I'm currently doing but after some considerations and a little bit of future-proof planning I want to opt for a Rack Storage Server. I started checking out HPE, Lenovo, Dell but the licensing, proprietary drives/hardware it's really confusing.

I already have a great network with some 10 Gigabit fiber to the computers and my switch so I would like the server to be able to use fiber.

I guess what I'm asking is for guidance when selecting hardware that can at least support 80TB and be somewhat good and future proof.

I tried googling for options but there's nothing concrete on how to do it, tried as well reaching out to a local business that specializes on building data centers but they quoted almost $2,000 USD only for licenses / warranty & specialized support that I do not want, it made me think that maybe if I want a rack server I need to pay for this???

I'm willing to learn what is necessary and spend accordingly. I have a max budget of $7,000 USD, I'm willing to throw extra money if needed for future proofing, so any recommendations are welcome

I’ve got a good offer(to me) on a r730xd, with 256GB of DDR4 ram, intel arc a310, dual 10Gb+dual 1Gb NIC. x2 E5-2666 V3.

This machine will see very ram dependent docker containers, the biggest selling points for me is the intel arc for my Plex transcoding. And the ram for my other container usages. I’ve already got 16TB disks, SSDs for cache. I use UnRaid Pro.

The other option is upgrading my current system to an i9-14900K, 48GB ram, Asus mobo on a tower I have everything else on (minus the GPU since the iGPU transcodes Plex great).

I just greatly need more cores and more RAM but the cores only need to be comparable to the 8700K I’ve been using, and the Xeon is just that.

They’re both comparable in price initially until I try to match the ram of the i9 system. Then I’m going above by at least $300.

Performance wise the i9 takes the cake every day and has the core count I’d need.

What would you do.

For people that build a home data center here, do you guys build first and then figure out the economics later? Or Is there a plan on how to use or sell the space first and then build accordingly?

In my Naive mind, there are 4 ways to profit:
1. Sell as a colocation place
2. Mine Crypto
3. Sell computing power for AI computation (not sure who will buy)
4. Sell VPS or web hosting

from these 4, mining crypto honestly sounds like the easiest option albeit being the riskiest.

Colocation/VPS feels like more of a marketing problem rather than a technical one. Not sure how people will buy the idea of hosting their potentially important stuff in a small scale data center. Maybe we can compete in price, but pretty sure that it's impossible to provide a competitive SLA.

Sell as AI computation power is just my logic telling that with all these new AI services, they must need an affordable computing power from somewhere right? How do we get them to buy the computing power from us?

I know that some of the people here use it for their existing business. And I know that some people don't even care about making profit. But am I looking this from the right perspective here?

What are people using for offline backups? I generate about 20TB/wk for work. Currently, I spit the data to a 104TB (usable) ZFS volume on a Supermicro then power down. What's the current data center tech?

Note: USB hard disks are not a suitable answer.