I created a file tree that looks similar to my system’s file tree, except it only contains all the files that I modified or added and only their respective directories. From there I just use rsync to sync those files/file tree to the system’s /.

It’s convenient to see what changes I currently have but it requires a bit of manual maintenance. I only really started doing it that way because I was learning how to use rsync and I just kept going on with it because it was working for me.

I’m only working with my laptop, android phone and two Raspberry Pi’s so I can get with my little rsync based setup.

True.

My self-hosting strategy is wildly alternative and not one I speak much about publicly. I’m the only person connecting to my own domain so as long as I continue to practice shutting the fuck up, I can get away with using multiple layers of obscurity rather than fiddling with third party solutions.

I check my logs daily and the only activity I ever see is my own. Since I am not hosting anything critical or sensitive, I have the opportunity to experiment this way without much risk to myself.

The way I’m set up, I am not concerned with DDOS attacks because it would fail to get past the Dynamic DNS. If I were hosting a social media platform or something more public, then I would need to take stronger measures to protect myself and that data.

Even though I don’t host anything important, I’m still glad I found alternative ways to hosting my own stuff without the use of any of Cloudflare services.

I’ve noticed over time that the self-hosted communities have been suggesting Cloudflare Tunnels less and less since Trump and his gang took over America. Maybe this latest outage will push more people to not recommend Cloudflare again in the future.

I still remember when I first got into self-hosting and being mocked pretty hard for questioning the use of such a large centralized service like Cloudflare. I’m glad I persisted and kept learning in my own direction but that still was very demotivating at the time.

I’ve experienced gatekeeping issues long before I got into self-hosting specifically. Years ago I wanted to learn C++ for Arduino and I was constantly talked down for asking questions.

“Why don’t you just do …” in response to a question feels very rude as a newcomer because it feels like I am being talked down to for not knowing what others already know. Even when I made an effort to show I was making an effort to learn on my own, I was still belittled.

I’m all for hearing different ways of approaching my issue but from the replies, it often feels like other people insist there is only one true specific way to handle an issue.

When I first got into self-hosting, people kept pushing Cloudflare on me. When I expressed concern over a large centralized corporation having that much control and how they might have service issues, I was mocked really hard. Half a year later and there was a significant outage and suddenly there’s all this talk about how centralized the internet is and how that is bad.

After that I took it upon myself to find alternative ways to protect myself without Cloudflare’s services but every step of the way has been an isolating experience. Every step of the way has been full of people saying that my efforts are pointless and that the bots will win anyways so I shouldn’t bother.

I decided to try to secure myself through multiple layers of obscurity and every question in that direction has been full of people saying that obscurity is not security, the bots will find you anyways!

I’ve stopped myself from asking too many questions now. I still keep learning in my direction. I feel like I’ve managed to find multiple solutions that both obscure and protect myself. I’ve constantly check my logs for months now and the bot is less than I expected in places I expect them to be and completely zero in other places I thought there would be some activity.

I want to share what I have learned and my experiences but I know I will receive backlash for deviating from the norm.

I’ve spent a lot of my self-hosting efforts trying to find ways to protect myself with minimal use of third party services, documenting as much as I could only feel afraid to share what I have learned.

This comment may not be about learning self-hosting as a beginner specifically but the vibe has been pretty damn consistent throughout me learning C++, self-hosting, linux and shell scripting. All things I enjoy but all so full of people ready to talk down to someone who wants to learn.

My web facing server has just enough packages installed to (kinda securely) host a Caddy and Kiwix docker container to work with my domain name and make a comfortable work environment through SSH. My Pi for my HomeAssistant docker container has less because it’s locked down to just my local network.

I also wrote my own install scripts so reinstalling everything and getting it back to a running state would take about 15 minutes for each device.

And I also wrote my own backup/restore scripts that evolved over 3/4 of a year. I use them often so I have confidence in those scripts.

I personally don’t really care too much. I have multiple ways of dealing with issues for something that’s a hobby to me. Which is why I stick to simplicity.

I’m sure this is a thing for people to worry about when dealing with more complex setups. I just wanna vibe out in my tiny corner of the internet.

On Alpine Linux I update my two Pi servers at 2 in the morning daily. It’s simpler compared to Debian which needs unattended-updates. Just add apk update && apk upgrade to a cron job and you’re good to go.

I only have three docker services which is simple enough to update manually.

I like to keep things as simple as possible for my already chaotic brain.

Thanks for the insight. It’s useful to know what tools are out there and what they can do. I was only aware of nmap before which I use to make sure the only ports open are the ports I want open.

My web facing device only serves static sites and a file server with non identifiable data I feel indifferent about being on the internet. No databases or stress if it gets targeted or goes down.

Even then, I still like to know how things work. Technology today is built on so many layers of abstraction, it all feels like an infinite rabbit hole now. It’s hard to look at any piece of technology as secure these days.

Early when I was learning self hosting, I lost my work and progress a lot. Through all that I learned how to make a really solid backup/restore system that works consistently.

Each device I own has it’s own local backup. I copy those backups to a partition on my computer dedicated to backups, and that partition gets copied again to an external SSD which can be disconnected. Restoring from external SSD to my Computer’s backup partition to each device all works to my liking. I feel quite confident with my setup. It took a lot of failure to gain that confidence.

I also spent time hardening my system. I went through this Linux hardening guide and applied what I thought would be appropriate for my web facing server. Since the guide seems more for a personal computer (I think), the majority of it didn’t apply to my use case. I also use Alpine Linux so there was even less I could do for my system but it was still helpful in understanding how much effort it is to secure a computer.

I was staying at a hostel in Sydney and It was just me and English girl just chilling in the common area. It was sort of exposed to the outdoors since there was no doors, just an entrance. It just lead to the sheltered outdoor area but each dorm room had their own heavy door.

We both found out that day that the big roaches that roam there knew how to fly. Not well. Like it struggled to carry it’s own weight. We both had time to react and do something. We both just watched in horror as it flew right into her hair.

podman ps shows the following:

CONTAINER ID  IMAGE                                 COMMAND               CREATED         STATUS         PORTS                                                         NAMES
daae60bdcc65  docker.io/library/caddy-caddy:latest  caddy run --confi...  47 minutes ago  Up 47 minutes  0.0.0.0:80->80/tcp, 0.0.0.0:5050->443/tcp, 2019/tcp, 443/udp  caddy

netstat -tunpl shows the following:

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:5025            0.0.0.0:*               LISTEN      3270/sshd: /usr/sbi 
tcp        0      0 0.0.0.0:5050            0.0.0.0:*               LISTEN      7342/conmon         
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      7342/conmon         
tcp        0      0 10.89.0.1:53            0.0.0.0:*               LISTEN      7336/aardvark-dns   
tcp6       0      0 :::5025                 :::*                    LISTEN      3270/sshd: /usr/sbi 
udp        0      0 10.89.0.1:53            0.0.0.0:*                           7336/aardvark-dns 

The only difference for the netstat command between Docker and Podman is that Podman show’s entries for aardvark-dns and Docker does not which is something I expect.

I sat down and managed to get wildcard certs working.

I figured I would leave my Caddyfile here in case anyone in the future needs a working reference. This is based off the Caddyfile mentioned in the original post.

Caddy Reference

# GLOBAL ENCRYPTION - DESEC.IO
{
        acme_dns desec {
                token "DeSEC.io Token Number"
        }
}

*.samplesite.ca {
        # SITE WIDE ENCRYPTION
        tls {
                dns desec {
                        token "DeSEC.io Token Number"
                }
        }
        # SUB DOMAIN #1
        @files host files.samplesite.ca
        handle @files {
                root * /srv
                file_server {
                        hide misc
                        browse
                }
        }
        # FALLBACK FOR UNHANDLED DOMAINS
        handle {
                abort
        }
}

Oh no, I was just about to move forward and then you gave me another rabbit hole.

I didn’t know Let’s Encrypt had a public database, that does sound like a good idea to use wildcard certs instead.

I assume this is what I can use as a reference for wildcard certs.

How do you keep track of probing? I’ve been curious about that but haven’t put much effort into that as I’ve been focused on getting things working.

That worked. I can finally see the padlock that says Verified by Let’s Encrypt.

I can’t believe all I had to do was wait. Thanks so much for sharing your experience.

Ah, it sounds similar to how I want to set up my headerless Raspberry Pi 5. I’ve been slowly learning tmux as well but it’s nice to have detached sessions. I’ll eventually add WeeChat as a sort of IRC bouncer for myself.

Right now it’s just hosting a simple file server and a copy of Wikipedia. I’ve also been looking at BashWrite and Bash Static Site Generator as a simple command line blog to host for myself.

I quite enjoy the text only work environment. It’s far less distracting :)

Yeah I don’t see any huge advantage in using a terminal browser over a full featured browser. However, I did notice that I don’t have to hide all those popup questions when I go to certain websites with troubleshooting questions.

Sometimes I just desire the reading the text without all the visual distractions that is present in our modern internet experience.

What was your experience with a terminal based linux setup? I imagine it as something extremely lightweight at the cost of convenience.

https://github.com/rothgar/awesome-tuis

It has a list of Text User Interfaces for terminals. Handy since I work in a terminal mostly with my network devices. A TUI for file management made life a lot easier for myself.

I can appreciate that about Debian. Common tools and stability can be both convinient and reliable. Learning linux is already overwhelming with choices.

Even though I use Alpine for all my Pi boards and laptop, I keep a live usb partition of Linux Mint Debian Edition as my emergency backup. It just works.

My frustration with Raspberry Pi OS is that the packages available were constantly out of date. Some were 2 to 3 years out of date.

I eventually started using Alpine linux on my Pi boards and have been happy since then. Now I can use the latest Docker and Podman packages without manually adding new repositories.

If I didn’t prefer Alpine’s minimal approach, I would have probably gone with Debian because of it’s history in stability.

I can’t remember a time where I didn’t have issues with syncing apps. They always just seem to stop working and it’s always frustrated me. It feels like to me there’s too much complexity under the hood and it exhausts me when something goes wrong.

As a result I’ve been using rsync commands as a way to sync files and folders. After the initial setup, I haven’t experienced any issues and it’s been consistent and reliable. I even created my own tool which simplifies each rsync command into a file for easier syncing/restoring of my data. I use it to sync between my laptop, android phone using Termux terminal and my two Raspberry Pi’s.

Rsync does have limitations like not being able to do two-way syncing but for my usage I am okay with that. I’m generally backing up data in most cases. Rsync does offer more flexibility since it’s a command line tool so I’m able to create backups of my entire system or sync specific files and folders. I can also set up cron jobs to have syncs performed periodically but I mainly do all my syncing manually because I like to see where all my data is moving.

As long as all my devices have static IP addresses within the network and I don’t mess around with any directory names/locations, everything just works and I’ve had a lot less headaches managing my data and backups. I also have a lot more confidence in my backup/restore process when compared to when I used other syncing programs/apps in the past.

Last month I spent most of my time writing a long and complex rsync script and was looking for something like this. I skimmed threw a few articles already and I’ve already found a bunch of things that I thought about adding but had difficulties finding before while I was writing the script.

The script I wrote is fully functional but I had already planned to revisit and rewrite parts of it because I enjoy knowing my script is solid, simple and reliable. But right now I have a couple other goals I want to finish before returning to my script.

This resource has lots to add on top of what I already learned.