Hi! After many years I finally pulled the trigger and am ditching google photos and going self hosting.
My main goal is to provide a safe archive for all my family photos, digitized VHSes, etc (around 2TB at the time, expected to grow, but not super rapidly). Since we are talking about literal tens of years of memories, I really want to do this properly, hence I ask for approval or hints what to improve. I obviously have a somewhat limited budget, and want to go as low on energy as possible.

What I'm currently setting up:

Main machine (Celsius w550):

This guys has like ~20-25W idle power draw. I can put maybe even 6x 3.5inch HDD without much hussle and the drives are ventilated very well.

Specs:

• Xeon e3-1220v5
• 16gb unregistered ECC ram
• Some random APC UPs that has not arrived yet
• GT 710 because I need video output and maybe for some simple transcoding

Drives:

• "Important drives": 2x 4tb exos 7e8 in RAID 1 for critical stuff (bought them almost new with <30h powered-on-time. They have "DELL EMC ENTERPRICE CLASS" sticker)
• "Unimportant drives": 1x 4tb exos 7e8 (same as above) without RAID for non-critical stuff, if I decide to, e.g. run Jellyfin.
• 16GB Intel Optane M10 as boot drive

Off-site backup machine (Hp Elitedesk 800 G1)

This guys has like ~15W idle power draw, but only 1x 3.5inch HDD bay without any mods.

Specs:

• Some 4th gen i5
• 8gb NON-ECC ram
• No UPS

Drives:

• 1x 4tb exos 7e8 (same as above) without RAID for syncing with "Important drives"
• Random sata ssd for boot drive.

The plan:

• Install truenas scale on both machines.
• Install immich on Main machine and store all my photos and videos on "Important drives"
• Sync contents of "Important drives" to my Off-site backup machine every week or so, using snapshots.
• Since off-site machine has no UPS, no ECC, no RAID I think I want to figure out a way to verify that contents of files on that machine and main machine match, every month or so.
• Don't bother syncing contents of "Unmportant drives", they are not very important.
• Setup the systems, so that the drives spin down to make them quieter and draw less power.

Questions:

• Is no RAID on off-site machine a horrible idea?
• Is verifying contents of off-site backup and main machine a common/good practice?
• 3-2-1 says two types of storage. I only have one. Should I care? If so, what do you recommend?
• Immich is sort of experimental from what I understand. Should I avoid it for my important photos?
• Does this setup even make sense? Any other hints?

Thanks in advance for your help, I really don't want to screw this up.

I've been steadily de-Googling for privacy reasons, and self hosting what I can over the last few months. YouTube was one of the last services I used because there are a lot of content creators that only upload there.

Yesterday I discovered Invidious and got it hosted and I'm really impressed! It's an alternative frontend for Youtube that strips the trackers and ads out. You can also make a private account on your instance, and import your YouTube subscriptions, so you get a subscription feed, watch history etc all synced on your devices without having to log into Google which I really like.

I am running it in my Gluetun stack so it uses my VPN's IP address to request videos from Google, rather than my public one, which is another layer of abstraction to help prevent tracking.

I'm using the Clipious app on Android phone and TV which syncs nicely with the desktop webapp.

Obviously I only access my instance over Tailscale: hosting a public instance might open you up to legal issues (although if its behind a VPN, I guess maybe not?), bandwidth throttling by Google, and obviously other people using my limited internet bandwidth to stream videos.

I want to use my first post to share my 10 year self-hosted journey.

Ten years ago, I lived in a student apartment that was terrible for someone loves the internet: there was no WiFi coverage and the power cut off every night after 10 PM. Each desk was assigned an 24/7 Ethernet port, however, the download speed was like ~10MB/s billed per GB.

During that time, portable drives were still quite popular in China. While their major use case was to carry some textbook-related materials for printing/presentation, students also exchanged some... precious Linux ISOs in person as dessert after dinner. I myself had a 512GB portable disk that stored around 200GB of such things, including some distros that people wouldn't recommend for children (I mean Arch or Linux with BTRFS before 2015). I often copied some of those to my phone at a really poor, unsatisfying, sdcard level of speed and conducted a meticulous examination of those content at bedtime.

This period marked the beginning of my journey into self-hosted solutions. I bought a Raspberry Pi and started using it as a WiFi access point. It worked smoothly and to some degree reduced my mobile bill. I soon realized that with a mobile power bank, I could have night WiFi coverage. Later, I discovered that the VLC app could directly play content from a network device using the SMB protocol, transforming my setup into a literally plug-and-play NAS. This setup gave me a very positive experience, which encourages me to always challenge things with self-hotsed solution.

Graduated from that university, I moved to the US for a master's degree. Despite having fast internet and full power/WiFi coverage, I faced a new challenge: accessing videos that required a China IP. This was particularly difficult since VPN software would not work due to the invisible wall known to every Chinese student aboard. Solutions like Shadowsocks were server applications, but my Raspberry Pi in China was in a wireless network backed by a NAT'ed ISP. How could I use it for this purpose?

I eventually used a VPS as a gateway and developed a solution quite similar to Tailscale. Its transport layer evolved multiple times, starting from TLS, DTLS (UDP version TLS), QUIC, KCP, to a home-made custom protocol for better performance in high latency network. The solution is not only for this purpose, but also for reducing the complexity of direct peer discovery and handling partial network failure e.g. one of the two routers is offline.

Now, I live in California in a small room with fast but not very stable internet. Working at a tech company allows me to invest more in my hobby as well as gain more knowledge about distributed systems. I subscribe to a Hetzner dedicated server located in Finland, set up a NAS at home, and run a small SBC server with 2x2TB drives at my parents' house in China. Now I have new challenge: persistently store important content, including photos, videos, code, and hard-to-find internet contents. Additionally, my parents, who believe in traditional Chinese medicine, needed to store numerous videos.

It's said that redundancy is the only way to reverse the increase of entropy. Silent bit rot, weird application behavior after months or years of server uptime, followed by sudden disk failures after a casual reboot are nothing surprising. Replication is necessary and geo-replication is better, but handling version conflicts, due to the nature of CAP in bidirectional sync, can be very hard, unless you know what can be sacrificed. While the concept of RAID is cool, is it truly useful for backups (rather than high availability)? Another hidden cost lies behind recognition, or to say it more clearly it is the maintenance load: How do you map your folder level replication/encryption/backup schedule configratuions to your whole dataset? Do you still remember it say 5 years later?

Those questions are somewhat opinion based, but I designed my own file system as the answer to those questions. My FS performs geo-replication, handles versioning and consistency, and runs on ZFS to provide read validation features. With this setup, I reached an extraordinary milestone: storing 3TB of content ... spreaded in 37TB of storage. The development process is quite pleasant and challengable and I recommend everyone to come up with their own solutions if time is allowed.

Combining the VPN and file system I developed, I created a quite stable system. I'd like to thank to the manufacturers (G* and K*) for their obviously mature and high quality mesh WiFi and HDD products, which made me consider automated failover e.g. network partitioning and disk failure more carefully and deeply.

Now, I have a better yet similar experience to when I first delved into self-hosted solutions back to 10 years ago — maintaining a server to play videos. The difference is that now, I'm more interested in problem-solving than the problem itself.

I love problem solving, that is why I love self-hosting. I know B2, Google Drive, combined with whatever somehow very secure VPN can achieve the same purpose in a ... much lower cost, but that is the point of self-hosted as a hobby, right?

• You can directly update an item in progress from the dashboard. This would save you a total of 6 clicks and 10 seconds of frustration when you just want to quickly update your reading progress.
• The browser no longer becomes unresponsive because of long lists (eg: show seasons, watch history, podcast episodes).

Screenshots: https://imgur.com/a/plgT5aN

About Ryot

Ryot aims to be a self hosted platform for tracking various facets of your life – media, fitness etc.

Links: Github and Discord.

Any suggestions or feature requests are welcome!

Hi!

I would like advices on what I want to do :

• File storage including photos and videos
• be able to consult with windows
• if possible through secured temporary access give access to family on specific folder
• eventually be able to host game server
• why not having a web gui to manage and or install plugin for photo video

I have a J6412 device with 8 gb ram and 2 x 4 TB hdd.

What OS to install in order to be as simple as possible ?

Scenario

Suppose you have a mesh VPN of >10 nodes using your favorite solution (be it manual wireguard, innernet, tailscale or sending pigeons with ciphertext). Suppose one of these nodes is your workstation and one is an application server on your LAN. Suppose you connect to this mesh network from your workstation or from mobile and tablet devices while on the go.

Now suppose you want to run a DNS server on your mesh network to map machines to human-readable hostnames and, while you're at it, to filter ads and third-party telemetry.

Suppose, too, that you like performance and optimization and the 18ms round trip to your off-premises mesh coordination server offends you on a personal level.

Question

You know that you can deploy DNS resolvers on all your nodes (or on all the nodes you care about and will connect to) and replicate zone data across them with AXFR/IXFR. But how do you tell your clients to pick the best (lowest latency) DNS server at run-time?

Solutions (or not)

Your mesh control server can advertise one or more DNS servers to all connected clients. This is how you'd normally do this. But you don't want to do things normally. You want lightspeed DNS response thank you very much, and 18ms is not fast enough to make the Kessel Run in 24 parsecs, let alone 12.

You can host a DNS server on your LAN application server. Sure, this gives you fast response from your workstation, but what about your phone, tablet or other nodes on your mesh? If you're in the office you certainly don't want to be pinging back into your LAN to pick up DNS responses before venturing out into the internet. You'd want to go through an off-premises server in the same city. But, while you're at home on the shitter with your phone, you equally don't want to be pinging some cloud server with 18ms latency while downloading some brownware. So it's not like you can define an off-prem DNS server for all clients and override this for your workstation.

You know that the way this is achieved in The Real World™ is with Anycast IP addresses. But you also know that Anycast won't work on a LAN and certainly not with whatever routing your mesh control server uses.

Now you might be forgiven for momentarily thinking of DNS-over-HTTPS and having the DoH hostname resolve to whatever would be the most convenient IP address based on the context before remembering that DoH is slower than DNS-over-UDP because of the TLS handshake and the necessity to have something to do the resolving in the first place.

So, you're back to thinking....

A clunky, hacky, stupid solution

One way to do this would be to combine multiple methods:

• No mesh-wide DNS. Nodes are hardcoded to use the closest regional DNS resolver.
• For the workstation, this means connecting to the LAN apps server.
• For mobile clients, you can run two nearly-identical mesh contexts, one that points to the LAN DNS resolver and one that advertises a DNS resolver on the mesh control server. Then use the mobile OS scripting features to automatically switch VPN contexts when they change network connections.

This works. But there has to be a more elegant weapon for this more civilized age. Any suggestions?

* ^{DNS is all well and good, but it is not the only service that could benefit from having lower latency on demand}.

https://once.com/writebook

37signals has released a new selfhosted application called Writebook. "Writebook is remarkably simple software that allows you to publish text and pictures in a simple, browsable online book format."

I have some uses for this (both at work and personally), so I figured I thought I would share here.

Hey guys,

im looking for a nice way to manage the Patching of my Linux Servers.

So far i only found Ansible + Semaphore, which looks a bit too complicated to me and ManageEngine Patch Manager, which is closed Source and only free for up to 25 Devices.

What do you guys use for that? A WebUI would be nice.

Howdy Wranglers, Noah here from https://receiptwrangler.io/.

We'll go over the development highlights of last month, and dive into what's coming this month.

Development Highlights:

V5 Release: Last month, version 5 of Receipt Wrangler released, check out the post here.

Mobile App Development (Unreleased): Last month, the ability to add/remove categories, and tags was added, as well as the ability to manually add shares to receipts.

Coming up in July:

The general theme of July will be finishing up the basic functionalities of the mobile app, and getting it ready to exit early access, as well as fixing up some bugs coming out of the V5 update.

Manually Adding Receipts: Users will be able to add receipts manually, just like in desktop.

Filtered Receipts Dashboard Widget: Users will be able to see their filtered receipt widgets like in desktop.

Receipt Sorting/Filtering: Users will be able to sort and filter their receipts like in desktop.

Receipt table view option: Users will be able to view their receipts as a table, instead of just as a list.

All in all, this month is exciting as it is bringing more feature parity to the mobile app in preparation for the first release which will be sometime next month.

Before the full release there will be a few more version to get released to Android/IOS so keep an eye out to for those to get the latest features.

Cheers!

Noah

Where do i get a small vps to run v2ray server, nothing fancy, I don't a lot of processing power or something.

I can't use oracle free tier or azure for free because I don't have a criedt card, i only have debit card

maximum to pay : 2$/month

Hi all,

some of you might have seen my previous articles and for those who haven't: I am on the journey documenting / creating guides on my home lab setup. The idea being, that you can follow my articles step by step to setup (or improve) your own home lab for self hosting purposes. This is my way of giving back to the community I have learned so much from.

To stay safe with our home lab, we also have to carefully plan backups. I just posted a guide on my strategy, which is a hands-of-and-forget-about-it approach.

It also includes running restic backups from TrueNAS Scale and connecting Proxmox to Healthchecks.io for monitoring.

I hope some of you find the article helpful and that it helps you to develop a save backup setup. Goal should be, that your mind can be at easy!

Enjoy: Backup Strategy with Restic and Healthchecks.io

Does anyone know a domain registrar that provides forwarding over SSL (HTTPS redirects)? This was so easy to implement with Google Domains, but I have not been able to do it elsewhere (i.e., Squarespace or Namecheap).

I've been self-hosting some internal/personal utilities (Plex/*arr, Home Assistant, etc.) for a few years now. I work from home, and primarily use my company issued laptop day-to-day including managing certain aspects of my homelab. The problem I ran into when I started this job and was assigned this laptop was that my internal subnet just so happens to ALSO be a subnet in use by my employer's network, so when I am connected to the corporate VPN, I would not be able to access my local network, since all requests to that subnet were routed through the VPN. I have had a few ideas/solutions to this issue, and was able to get access to internal (Accessible by me only) services, but am still struggling with my Wordpress site that I sometimes share posts from to friends and family. The options that I came up with so far are:

1. Accessing the site from the public IP - I am using an ISP provided router that has NAT Loopback disabled and give no way to enable it.
2. Static routes on my laptop - This was working great for everything UNTIL the IT department stripped my Administrative rights from my laptop.
3. Cloudflare - This works great for internal services but requires some sort of authentication provider to be used to access the site. I'm trying to avoid having my users need to add an additional form of authentication to the site.
4. Separate URL (Routed to Cloudflare) - As far as I know, Wordpress doesn't allow multiple URLs to be used, so it's kind of a one or the other situation.
5. Access via web proxy - IT has web proxies blocked.
6. Disconnect from VPN every time I want to edit/access the site - This is just plain annoying.
7. VNC to another machine - This is my current 'solution' to the problem. I have VNC enabled and access it through my Cloudflare Tunnel to update the site.
8. Re-Subnet my network - This is IMO probably my best, most complete solution, but will be a pretty big project that I just don't have the time for at this time. I am working on getting the gameplan together to make sure I do it right, and without disturbing the wife and kids.

Does anyone know of another way that I could get access to my site from my work PC without Public or Internal IP access for free until I can get my network re-subnetted? For reference, my current infrastructure related to this is as follows:

• ISP provided router that doesn't support NAT Loopback, or multiple subnets.
• Two docker hosts running Ubuntu.
• Two Unifi AP's
• My work laptop, with no admin access, with a VPN that routes traffic on my local subnet through it.

Hi,

I want to migrate my mail server (plain postfix, dovecot, etc.) to a new solution with a good UI.

Apart from the standard features (like multiple domains, DKIM/DMARC/...), I also need custom transports for relayed domains (accept all recipients and relay it to the transport server).

As for certificate handling, ideally, I'd have a Traefik reverse proxy in front with the proxy protocol enabled and TLS termination for all ports because I'm considering hosting SimpleLogin on the same server (now on a different server).
I also plan to use IaC (Pulumi) to manage the server, which ideally requires an automated update process through the dockerized image.

So far, I am contemplating these solutions:

• Mailcow:

  • looks pretty solid with a vast feature set
    
  • seems pretty memory intensive
    
  • documentation specifies that I must make the certs available to it which I don't really get why if Traefik would handle all TLS termination and even if I'd mount the Traefik certs (how does mailcow know they got renewed?)
    
  • upgrades seem to involve manual commands

• Mailu:

  • seems to have a slightly smaller feature set as Mailcow but covers all my needs
    
  • apparently less memory intensive
    
  • I can use the notls option, hoping this will work with Traefik in front (does it?)
    
  • upgrades seem to involve manual commands

• Stalwart:

  • pretty lightweight
    
  • no virus scan
    
  • works well with Traefik
    
  • manual upgrade process with basically emptying the database, which makes upgrades inherently manual and, atm, "useless" for me
    
  • i really like the modern approach and being able to store emails in S3 for an easy backup

What are your (long-term) experiences with these solutions? How do you backup your emails?
I'd also be grateful for your take regarding my requirements, especially certificate and upgrade handling, regarding these. :)

I am currently dipping my toes into storing blu rays big hard drives. Before committing to an actual build I want to test things on a spare laptop I have laying around. I’ve seen a bunch of videos but I’m not sure if I will waste a ton of time following those tutorials , since all of them either focus on gaming server or media server.

I’m already running Linux on that laptop.

I’ve heard of casaOS being a solid choice and I can install jellyfin from there. Is that the best option right now ?

And when I want to also host a gaming server, can I add to that setup or do I need a whole different basis? CasaOS doesn’t seem to be gaming focused.

Or can those applications run next to each other and I just install whatever gaming server I want in the Linux cmd?

I feel like I’m only missing a thread connecting all of the information I already gathered.

My laptops power or specs don’t really matter here, I’m not planning to actually host gaming servers on that laptop, but I want to test around to see how things work before ordering dedicated parts for a server.

Hi all,

I use Nextcloud as my primary calendar, and link it to iOS via the iOS CalDAV in settings.

However, with most calendar apps, the calendar update is slightly out of sync so is generally reliable. For example, if I add something via Nextcloud on my laptop, it could take 30 mins+ for my phone to sync this.

Does anyone know any iOS calendar apps that handle CalDAV natively inside the app instead of relying on the iOS calendars themselves?