Hi All,

TL:DR at end.

I was manic a while back and had a great idea to build a home datacenter (this was before I met y'all) so I could learn how the cloud works better. I am an instructor at a technical college, but I've always focused on the analysis/presentation side of data work. Perhaps unsurprisingly, a data scientist can do cool stuff but not this. I was/am hoping to develop curriculum for a new course for those interested in either data-center work or using the cloud in general.

To that end, I'm hoping to talk to experts in basically every aspect of the data-center (infiniband, RDMA, RDMAoPCIe, PCIe networking in general, orchestration, defining workflows, security, etc...) at a scale that would fit on a benchtop or I could at least have control over the components and switch configurations as necessary. To that end, I have a bunch of small x86, Jetson (ARM), and Bluefield (ARM+NIC), Broadcom PCIe switch, and Infiniband router systems I was hoping to play with -bought mostly secondhand.

I'm hoping if I occasionally post questions about my goals in spinning this thing up I can get some feedback, suggestions, and critiques toward getting the construction of the physical layer stable. I know I'm doing it wrong because peak functionality is normally the goal and this is more about demonstration of the various technologies involved than an optimization problem (that would require me to circle back to my current class and I am not ready to introduce them to this yet, not while I still have no idea what I'm doing!)

I need guidance around what a reasonable entry point looks like given what I have and my thoughts vs the reality of what the data center is like today (which I have no vision into). Please, I don't think I'm asking for forbidden knowledge, but it sure feels that way.

TL:DR, may I ask dumb questions and hope for smart answers?