See build Post -- Advice/Discussion: Running Local LLM's - Builds : r/homelab

This might be a longish post:

I've been really toying with the idea of running a local LLM or two.

idea for use cases (most of this was experimental)-

• private ChatGPT for the family and kids and keep data private. but would match gpt-4 in speed or get close to it.
  • have guardrails for the kids in the house (at least experiment with it)
  • Have AI "evolve" with our household until my kid gets into high school or longer. Toddler currently.
• have AI running and processing (6) 4k security camera feeds and with LPR and face detection, animal detection/possible identification (i live in an area with a lot of animals roaming around)
• replace siri and redirect to my own voice assistant for the house. (experimental)
• OPNsense log analysis for network security
• Photo/Media/Document organization, (i.e. themes, locations, faces, etc.)
  • goal of moving all media to a local personalized cloud and out of the actual cloud (at some point)
• Future - possible integration of AI into a smart home. (using camera's to see when i pull up and get the house ready for me as i get out.... sounds cool)
• Using a magic mirror for something (cause it sounds cool, may not be feasible)

With the Mac Studio Upgrade 512gb of unified memory seemed like it would be a pretty legit workstation for that. I got into a discussion with ChatGPT about it and went down a rabbit hole. Some of the options was to create a 2 machine (all the way up to 5) Mac Studio cluster using Exos then connecting the nodes through a 200gbe (to obviously reduce latency and increase token processing) NIC in a peer-2-peer setup, connected to thunderbolt via an eGPU enclosure.

As I said rabbit hole. I've spent a number of hours discussing and brainstorming, pricing and such.

The hang up with the Mac Studio that is making me sad is that the video processing and most of the realtime processing is is just not there yet. The unified memory and system power efficiency just doesn't make up for the raw horsepower of nvidia cuda. At least compared to having a linux server with a 4090 or 4080 and room for 1 or 2 more gpus later down the road.

Here's the Linux builds that ChatGPT came up with. Listing so that people can see.

See build Post -- Advice/Discussion: Running Local LLM's - Builds : r/homelab

I say all that to ask the community in a discussion format.

• Has anybody tried any of this? What was your experience?
• Is the Mac Studio even remotely feasible for this yet, (because MLX acceleration is not fully implemented across all models yet.)
  • Has anybody tried to process 4k video streams in realtime for AI recogonition? Does it work?

See build post-- Advice/Discussion: Running Local LLM's - Builds : r/homelab

Whew, typing all this out, man this is ambitious. I do realize i would be doing all of this 1 at a time, honing and then integrating. I can't be the only one here that's thought about this.... so my peeps what say ye.

Hey everyone,
I really appreciate the content provided in this thread! I need some input on my setup. The closet all my equipment is in stays at a around 85 degrees. My servers and disk stay with in 40C range. Do you think, long term, this is to high of a temp? I currently have an AC infinity fan sucking the hot air out of the room with an AC infinity fan on the door to pull air. Any input is greatly appreciated!

I have an APC UPS that require standard C13 cables. I am getting a decent PDU off a friend but that PDU has a C20 input.

Can I use a C19 to C14 able? No issues? Have never done it like this because PDUs I worked with were connected to bigger UPSes so no problems there

Hi! I'm looking to purchase several Dell R740 or HP DL380 G10 servers and need recommendations for reliable US-based sellers. I'm already familiar with newserverlife.com and techmikeny.com, but I've only bought from newserverlife.com so far.

Thank you

I noticed that eBay often beats Amazon’s prices for storage and server parts, but spotting the best deals can be really tough. eBay's interface isn't built for that purpose, and pricing, stock, sellers, and discounts change by the minute.

So, I built a tool that replaces the entire search experience when it comes to Storage and Server Parts. Whether you need a single SSD, a pallet of HDDs, or anything in between, the goal is to make finding deals effortless.

It now supports eBay in multiple regions: USA, Canada, Australia, and major European markets (UK, Germany, France, Italy), and has a built-in currency converter.

What Makes It Efficient?

• Uncovers Trending Listings, Hidden Coupons & Bulk Sales: The tool has a "Rapidly Selling" sorting option, which shows what’s trending so you can see which listings are currently attractive to other buyers in your region. Often, this also means the listing has hidden promos (coupons or multi-quantity discounts), so the actual price is even lower.

• Cost per TB & Total Capacity (for any storage device): You can sort all storage devices by price per terabyte and filter them by total capacity, it’s highly accurate, even for bulk lots.

• All Prices Include Total Cost (Including Domestic Shipping): See the real price upfront, including shipping costs within each eBay market.

• Real-Time Listings: New listings get added constantly, so you don’t miss fresh deals - some listings only last hours or even minutes before selling out.

• Listings Info & Seller Feedback at a Glance: There’s no need to sift through every single listing - the tool highlights useful details about the item and seller upfront. You can read relevant seller reviews with just one click, without having to navigate away.

• Supports SSDs, NVMe, & Enterprise Storage: It’s not just HDDs, it covers a wide range of storage gear and server parts.

• Mobile-Friendly: Works smoothly on phones and tablets, so you can browse it anywhere.

• Share or Bookmark Custom Searches: Every search has a unique URL, so you can save it for later or share it with friends.

Tips for Hunting Deals:

• Try Sorting by "Rapidly Selling" (for Any Type of Hardware) This shows listings that people are actively buying right now - often a sign they’re undervalued or include hidden discounts.
• Sort by Lowest Cost per TB (for Storage) If your goal is maximum storage per dollar, this highlights the best values instantly.
  • In some edge cases, the analytics might no be accurate, so If something looks too cheap, just scroll further to find more accurate bargains.
• Results from these two sorting options might overlap in some cases:
  • "Rapidly Selling" only includes multi-quantity listings, since those are the ones that can be tracked for ongoing sales.
  • "Lowest Cost per TB" highlights both multi quantity, single-unit, and single bulk-lot listings.
  • Sometimes they intersect: if a great cost-per-TB listing is trending, it will show up in both views.
• Use Keyword Filters for More Control
  • Sometimes Sellers often place items in the wrong eBay category.
  • To broaden or refine your search, clear all filters and use specific keywords (e.g., “SAS”, "2.5”, “10TB”, or a brand name).
  • Example: Some sellers mislist HDDs in the SSD category, so adding “SSD” as a keyword filter will clean up false results.

Recent Updates & Feedback

I posted an earlier version of this tool a while back and used the feedback to improve the interface, filtering, and global region support. After testing, I’ve found good pricing across all regions.

Would love to hear your thoughts or feature ideas - I’m open to expanding this to other eBay categories if there’s demand!

Check it out here

For anyone interested in training their own LLM's, here is something you guys might be interested in :) https://www.troostwijkauctions.com/en/l/nvidia-dgx-1-with-v100-2x-xeon%2528r%2529-e5-2698-v4-16x-32-gb-ram-no-hdd-rack-mount-chassis-A1-28034-3505

Hey everyone,

I'm working on getting my homelab cabling cleaned up and could really use some advice. I have a 24U rack, and as you can see in the pics, the Ethernet cables are a bit all over the place. The Rear is a mess as well.

Here's my setup:

10GbE/40GbE switch: Planning to use this for storage and vMotion traffic. (NOT in use yet)

1GbE network: Dedicated for IPMI and management interfaces.

Side panels on my rack make vertical cable management tough, so I need alternative ways to keep things clean.

What’s the best way to route and secure these cables so they look better and stay manageable? Would horizontal cable management help in this case? Any tips or products that you swear by?

Appreciate any advice! Pics attached for reference.

https://i.imgur.com/Y5irtOl.jpeg

https://i.imgur.com/HYNObGv.jpeg

Which is more reliable: NAS, cloud storage, or an external hard drive?

I've been having trouble with line interactive UPSes and my generator on my main PC. Frequently, when I transfer to generator the UPS will drop the load entirely with a wiring fault. Not every time, just about 80% of the time.

For my lab, I have a CyberPower OL1500RTXL2U (online UPS) and have had zero issues. I wanted to use the same UPS on my main PC but have been reluctant due to the fan noise. Someone posted an article a couple of years back on replacing the fans but there weren't any direct-fit alternatives so they ended up creating a shroud and mounting it outside the UPS. Not something I wanted to do.

Well, I went ahead and bit the bullet thinking I could deal with the noise and ordered a used one. I unboxed it, powered it up and was like, no fucking way is this going to work. It was way too noisy.

After a bit of google searching, I stumbled upon a (new?) Noctua product. It was the NF-A8 FLX. It's a 12V, 3-pin fan that's the exact same size as the PSAD18025BH used in the UPS. They were only about $16 apiece so I ordered two of them (the UPS has two, one in the front as a booster and one in the back).

Long story short(er), I wired up the back fan, red to red, black to black, and white to yellow (RPM sensor) and it worked; however, the UPS flashed FAN ERROR for a moment and it cleared. Even better, though, it was almost silent!

I installed the back fan and replaced the front fan as well (same deal), closed up the UPS and powered it up again. FAN ERROR, except it didn't clear this time. Powering the UPS off and back on again did clear the error and it hasn't occurred again.

I'm wondering if the FAN ERROR is due to the Noctua fans not spinning up as quickly as the stock fans. Powering the UPS off/on doesn't spin them down completely, so it may re-measure the RPMs and have it be within specs.

In any case, I'm now up-and-running with my new (used) UPS. It's under my desk and quieter than my gaming PC now. I have no idea whether the fan situation is going to come back and bite me, but it's working for now.

I have an APC SmartUPS 1500 UPS.

The 24V batteries are connected using 50A connectors, so that limits them to 1,200W DC. After conversion losses, that's going to peak at 900-1,000W.

They APC claim 1,500VA... how does that work? VA is a measure of the current and voltage, but as the voltage and current are nominally sine waves, they can be out of phase. If the current is 90' before or after the voltage then there will be no useful power transmitted, no matter the VA figure.

So, and I don't know the details, your UPS will probably be delivering a lot less power than you might expect.

How do I know this? I am upgrading the batteries in mine to 24V 100Ah, up from 24V 20Ah and realised that the Andersen connectors used were only 50A, which got me thinking!

Hello everyone,
I have a Dell C6400 with 4 C6420 sleds, and I need PCIe Gen 4 support. I’ve found that the C6520 and C6525 sleds are compatible with PCIe Gen 4, so I’m looking to purchase 2 C6520 or C6525 sleds to replace 2 of my current C6420 sleds.

Has anyone had the opportunity to test this sled configuration?
Thank you!

It's designed for people who regularly buy hard drives, servers, and other homelab parts on eBay, especially in bulk quantities.

Some of the features I thought might be useful:

• Shows new bulk listings across multiple categories
• Lets you see cost per TB, total capacity, and other useful info without opening every listing
• Lets you sort by what’s currently selling fast so you can spot trends
• Tracks price drops and flags sellers who are likely to accept lower offers
• Option to set real-time email alerts when something matching your filters gets listed
• Keyword search if you're looking for something very specific

You can check it out here to see the storage section, and if you clear the filters it will show other computing parts.

I’d love to get some feedback to see if this is actually useful to anyone here.
If there are features you’d want added, I’m happy to keep improving it based on what people need.

Has anyyone experience running U.2 SSD in a Z1 array under proxmox? How does that go performance wise?

I plan on using 3x Samsung PM9A3 3.84 TB disks for that per server. (mainly because i already have some).

Hey everyone, I'd like to know your recommendations for a 4port NIC capable of teaming (LBFO). Thanks!

I’ve been using the Ugreen NAS I got during Black Friday for a month now, and so far, it’s been a great experience. The most convenient part for me is how it saves space on my phone (iPhone users can probably relate to this struggle). My phone is packed with videos, and even though I got the largest storage option, it still feels like it’s never enough over time. But ever since I got a NAS, all those headaches are gone. After a month of use, I haven’t noticed any downsides yet. Anyone who's been using it longer? What's your experience been like? Any tips or tricks for making the most out of it?