@selfhosted@lemmy.world

Mid 2022, a friend of mine helped me set up a selfhosted Vaultwarden instance. Since then, my “infrastructure” has not stopped growing, and I’ve been learning each and every day about how services work, how they communicate and how I can move data from one place to another. It’s truly incredible, and my favorite hobby by a long shot.

Here’s a map of what I’ve built so far. Right now, I’m mostly done, but surely time will bring more ideas. I’ve also left out a bunch of “technically revelant” connections like DNS resolution through the AdGuard instance, firewalls and CrowdSec on the main VPS.

Looking at the setups that others have posted, I don’t think this is super incredible - but if you have input or questions about the setup, I’ll do my best to explain it all. None of my peers really understand what it takes to construct something like this, so I am in need of people who understand my excitement and proudness :)

Edit: the image was compressed a bit too much, so here’s the full res image for the curious: https://files.catbox.moe/iyq5vx.png And a dark version for the night owls: https://files.catbox.moe/hy713z.png

    • @7Sea_Sailor@lemmy.dbzer0.comOP
      link
      fedilink
      English
      41 year ago

      I’m still on the fence if I want to expose Jellyfin publicly or not. On the one hand, I never really want to stream movies or shows from abroad, so there’s no real need. And in desperate times I can always connect to Tailscale and watch that way. But on the other, it’s really cool to simply have a web accessible Netflix. Idk.

        • @7Sea_Sailor@lemmy.dbzer0.comOP
          link
          fedilink
          English
          11 year ago

          Gosh, that’s cute. Probably how I’ll end up too. Right now I’m not ready to let friends use my services. I already have friends and family on adguard and vaultwarden, that’s enough responsibility for now.

  • @ginko@lemmy.world
    link
    fedilink
    English
    131 year ago

    Architecture looks dope

    Hope you’ve safeguarded your setup by writing a provisoning script in case anything goes south.

    I had to reinstall my server from scratch twice and can’t fathom having to reconfigure everything manually anymore

    • @7Sea_Sailor@lemmy.dbzer0.comOP
      link
      fedilink
      English
      111 year ago

      Nope, don’t have that yet. But since all my compose and config files are neatly organized on the file system, by domain and then by service, I tar up that entire docker dir once a week and pull it to the homelab, just in case.

      How have you setup your provisioning script? Any special services or just some clever batch scripting?

      • @ginko@lemmy.world
        link
        fedilink
        English
        41 year ago

        Old school ansible at first, then I ditched it for Cloudbox (an OSS provisioning script for media server)

        Works wonders for me but I believe it’s currently stuck on a deprecated Ubuntu release

  • I’ve saved this. I set up unraid and docker, have the home media server going, but I’m absolutely overwhelmed trying to understand reverse proxy, Caddy, NGINX and the security framework. I guess that’s my next goal.

    • @7Sea_Sailor@lemmy.dbzer0.comOP
      link
      fedilink
      English
      211 year ago

      Hey! I’m also running my homelab on unraid! :D

      The reverse proxy basically allows you to open only one port on your machine for generic web traffic, instead of opening (and exposing) a port for each app individually. You then address each app by a certain hostname / Domain path, so either something like movies.myhomelab.com or myhomelab.com/movies.

      The issue is that you’ll have to point your domain directly at your home IP. Which then means that whenever you share a link to an app on your homelab, you also indirectly leak your home location (to the degree that IP location allows). Which I simply do not feel comfortable with. The easy solution is running the traffic through Cloudflare (this can be set up in 15 minutes), but they impose traffic restrictions on free plans, so it’s out of the question for media or cloud apps.

      That’s what my proxy VPS is for. Basically cloudflare tunnels rebuilt. An encrypted, direct tunnel between my homelab and a remote server in a datacenter, meaning I expose no port at home, and visitors connect to that datacenter IP instead of my home one. There is also no one in between my two servers, so I don’t give up any privacy. Comes with near zero bandwith loss in both directions too! And it requires near zero computational power, so it’s all running on a machine costing me 3,50 a month.

      • I appreciate this thoughtful reply. I read it a few times, I think I understand the goal. Basically you’re systematically closing off points that leak private information or constitute a security weakness. The IP address and the ports.

        For the VPS, in order for that to have no bandwidth loss, does that mean it’s only used for domain resolution but clients actually connect directly to your own server? If not and if all data has to pass through a data center, I’d assume that makes service more unreliable?

        • @7Sea_Sailor@lemmy.dbzer0.comOP
          link
          fedilink
          English
          31 year ago

          Your first paragraph hits the nail on the head. From what I’ve read, bots all over the net will find any openly exposed ports in no time and start attacking it blindly, putting strain on your router and a general risk into your home network.

          Regarding bandwith: 100% of the traffic via the domain name (not local network) runs through the proxy server. But these datacenters have 1 to 10 gigabit uplinks, so the slowest link in the chain is usually your home internet connection. Which, in my case, is 500mbit down and 50mbit up. And that’s easily saturated on both directions by the tunnel and VPS. plus, streaming a 4K BluRay remux usually only requires between 35 and 40 mbit of upload speed, so speed is rarely a worry.

          • @atzanteol@sh.itjust.works
            link
            fedilink
            English
            11 year ago

            bots all over the net will find any openly exposed ports in no time and start attacking it blindly,

            True.

            putting strain on your router

            I guess? Not more than it can handle mind. But sure there will be a bit of traffic. But this is also kinda true whether you expose ports or not. The scanning is relentless.

            and a general risk into your home network.

            Well…If your proxy forwards traffic to your home network you’re still effectively exposing your home network to the internt. There’s just a hop in between. Scans that attack the web applications mostly don’t know or care about your proxy. If I hacked a service through the proxy I still gain access to your home network.

            That said, having crowdstrike add a layer of protection here is a good thing to potentially catch something you didn’t know about (eg a forgotten default admin password). But having it on a different network over a vpn doesn’t seem to add any value here?

            • @7Sea_Sailor@lemmy.dbzer0.comOP
              link
              fedilink
              English
              21 year ago

              You make a good point. But I still find that directly exposing a port on my home network feels more dangerous than doing so on a remote server. I want to prevent attackers sidestepping the proxy and directly accessing the server itself, which feels more likely to allow circumventing the isolations provided by docker in case of a breach.

              Judging from a couple articles I read online, if i wanted to publicly expose a port on my home network, I should also isolate the public server from the rest of the local LAN with a VLAN. For which I’d need to first replace my router, and learn a whole lot more about networking. Doing it this way, which is basically a homemade cloudflare tunnel, lets me rest easier at night.

              • @atzanteol@sh.itjust.works
                link
                fedilink
                English
                41 year ago

                You make a good point. But I still find that directly exposing a port on my home network feels more dangerous than doing so on a remote server.

                You do what makes you feel comfortable, but understand that it’s not a lot safer. It’s not useless though so I wouldn’t say don’t do it. It just feels a bit too much effort for too little gain to me. And maybe isn’t providing the security you think it is.

                It’s not “where the port is opened” that matters - it’s “what is exposed to the internet” that matter. When you direct traffic to your home network then your home network is exposed to the internet. Whether though VPN or not.

                The proxy server is likely the least vulnerable part of your stack, though I don’t know if “caddy” has a good security reputation. I prefer to use Apache and nginx as they’re tried and true and used by large corporations in production environments for that reason. Your applications are the primary target. Default passwords, vulnerable plugins, known application server vulnerabilities, SQL injections, etc. are what bots are looking for. And your proxy will send those requests whether it’s in a different network or not. That’s where I do like that you have something that will block such “suspect” requests to slow such scanning down.

                Your VPS only really makes any sense if you have a firewall in ‘homelab’ that restricts traffic to and from the VPN and specific servers on specific ports. I’m not sure if this is what is indicated by the arrows in and out of the “tailscale” box? Otherwise an attacker with local root on that box will just use your VPN like the proxy does.

                So you’re already exposing your applications to the internet. If I compromise your Jellyfin server (through the VPS proxy and VPN) what good is your VPS doing? The first thing an attacker would want to do is setup a bot that reaches out to the internet establishing a back-channel communication direct to your server anyway.

                Judging from a couple articles I read online, if i wanted to publicly expose a port on my home network, I should also isolate the public server from the rest of the local LAN with a VLAN.

                It’s not “exposing a port that matters” - it’s “providing access to a server.” Which you’ve done. In this case you’re exposing servers on your home network - they’re the targets. So if you want to follow that advice then you should have your servers in a VLAN now.

                The reason for separating servers on their own VLAN is to limit the reach an attacker would have should they compromise your server. e.g. so they can’t connect to your other home computers. You would create 2 different networks (e.g. 10.0.10.0/24 and 10.0.20.0/24) and route data between them with a firewall that restricts access. For example 10.0.20.0 can’t connect to 10.0.10.0 but you can connect the other way 'round. That firewall would then stop a compromised server from connecting to systems on the other network (like your laptop, your chromecast, etc.).

                I don’t do that because it’s kinda a big bother. It’s certainly better that way, but I think acceptable not to. I wouldn’t die on that hill though.

                I want to be careful to say that I’m not saying that anything you’re doing is necessarily wrong or bad. I just don’t want you to misunderstand your security posture.

    • @Specal@lemmy.world
      link
      fedilink
      English
      31 year ago

      Caddy makes it so you don’t have to understand reverse proxies to use them, the config is literally just "reverse_proxy " and then gives it a let’s encrypt SSL certificate. It’s beautiful for self hosting.

    • jelloeater
      link
      fedilink
      English
      201 year ago

      You should see some of the literal data centers folks have in their houses. It’s nuts.

  • @krash@lemmy.ml
    link
    fedilink
    English
    101 year ago

    How do you like crowdsec? I’ve used it on a tiny VPS (2 vcpu / 1 GB RAM) and it hogs my poor machine. I also found it to have a bit of learning curve, compared to fail2ban (which is much simpler, but dosen’t play well with Caddy by default).

    Would be happy to see your Caddy / Crowdsec configuration.

    • @Sekki@lemmy.ml
      link
      fedilink
      English
      21 year ago

      Somehow I only had issues with CrowdSec. I used it with Traefik but it would ban me and my family every time they used my selhosted matrix instance. I could not figure out why and it even did that when I tried it on OPNSense without the Traefik bouncer…

    • Rob Bos
      cake
      link
      fedilink
      English
      11 year ago

      I have crowdsec on a bunch of servers. It’s great and I love that I’m feeding my data to the swarm.

    • @7Sea_Sailor@lemmy.dbzer0.comOP
      link
      fedilink
      English
      5
      edit-2
      1 year ago

      The crowdsec agent running on my homelab (8 Cores, 16GB RAM) is currently sitting idle at 96.86MiB RAM and between 0.4 and 1.5% CPU usage. I have a separate crowdsec agent running on the Main VPS, which is a 2 vCPU 4GB RAM machine. There, it’s using 1.3% CPU and around 2.5% RAM. All in all, very manageable.

      There is definitely a learning curve to it. When I first dove into the docs, I was overwhelmed by all the new terminology, and wrapping my head around it was not super straightforward. Now that I’ve had some time with it though, it’s become more and more clear. I’ve even written my own simple parsers for apps that aren’t on the hub!

      What I find especially helpful are features like explain, which allow me to pass in logs and simulate which step of the process picks that up and how the logs are processed, which is great when trying to diagnose why something is or isn’t happening.

      The crowdsec agent running on my homelab is running from the docker container, and uses pretty much exactly the stock configuration. This is how the docker container is launched:

        crowdsec:
          image: crowdsecurity/crowdsec
          container_name: crowdsec
          restart: always
          networks:
            socket-proxy:
          ports:
            - "8080:8080"
          environment:
            DOCKER_HOST: tcp://socketproxy:2375
            COLLECTIONS: "schiz0phr3ne/radarr schiz0phr3ne/sonarr"
            BOUNCER_KEY_caddy: as8d0h109das9d0
            USE_WAL: true
          volumes:
            - /mnt/user/appdata/crowdsec/db:/var/lib/crowdsec/data
            - /mnt/user/appdata/crowdsec/acquis:/etc/crowdsec/acquis.d
            - /mnt/user/appdata/crowdsec/config:/etc/crowdsec
      

      Then there’s the Caddyfile on the LabProxy, which is where I handle banned IPs so that their traffic doesn’t even hit my homelab. This is the file:

      {
      	crowdsec {
      		api_url http://homelab:8080
      		api_key as8d0h109das9d0
      		ticker_interval 10s
      	}
      }
      
      *.mydomain.com {
      	tls {
      		dns cloudflare skPTIe-qA_9H2_QnpFYaashud0as8d012qdißRwCq
      	}
      	encode gzip
      	route {
      		crowdsec
      		reverse_proxy homelab:8443
      	}
      }
      

      Keep in mind that the two machines are connected via tailscale, which is why I can pass in the crowdsec agent with its local hostname. If the two machines were physically separated, you’d need to expose the REST API of the agent over the web.

      I hope this helps clear up some of your confusion! Let me know if you need any further help with understanding it. It only gets easier the more you interact with it!

      don’t worry, all credentials in the two files are randomized, never the actual tokens

      • @krash@lemmy.ml
        link
        fedilink
        English
        11 year ago

        Thanks for the offer! I might take you up on that :-) If you have a Matrix handle and hang out in certain rooms, please DM me and I’ll harass reach out to you there.

        • @7Sea_Sailor@lemmy.dbzer0.comOP
          link
          fedilink
          English
          1
          edit-2
          1 year ago

          Hm, I have yet to mess around with matrix. As anything fediverse, the increased complexity is a little overwhelming for me, and since I am not pulled to matrix by any communities im a part of, I wasn’t yet forced to make any decisions. I mainly hang out on discord, if that’s something you use.

    • @RiderExMachina@lemmy.ml
      link
      fedilink
      English
      11
      edit-2
      1 year ago

      With the enshittification of streaming platforms, a Kodi or Jellyfin server would be a great starting point. In my case, I have both, and the Kodi machine gets the files from the Jellyfin machine through NFS.

      Or Home Assistant to help keep IOT devices that tend to be more IoS. Or a Nextcloud server to try to degoogle at least a little bit.

      Maybe a personal Friendica instance for your LAN so your family can get their Facebook addiction without giving their data to Meta?

      • @Specal@lemmy.world
        link
        fedilink
        English
        41 year ago

        Additionally, using jottacloud with 2 VPS’s (one of them being built on epyc like from OVH cloud) can get you a really good download server and streaming server for about £30 a month, which is the same as having netflix and Disney plus, except now you can have anything you want.

        I have a contabo 4core 8gb ram VPS that handles downloading content.

        A OVH 4core 8gb VPS that handles emby (I keep trying to go back to jellyfin but it’s just slightly slower than emby at transcoding and I need to squeeze as much performance out of my VPS as possible so… Maybe one day jelly)

        And I have a really good streaming experience with subtitles that don’t put big black boxes on the screen making 1/8th of the screen non viewable.

  • Cyanogenmon
    link
    fedilink
    English
    31 year ago

    This is oddly similar to some informal workups I’ve done for our work network.

    Nice work 👍.

    • @7Sea_Sailor@lemmy.dbzer0.comOP
      link
      fedilink
      English
      51 year ago

      its basically a VPS that comes with torrenting software preinstalled. Depending on hoster and package, you’ll be able to install all kinds of webapps on the server. Some even enable Plex/Jellyfin on the more expensive plans.

        • @7Sea_Sailor@lemmy.dbzer0.comOP
          link
          fedilink
          English
          5
          edit-2
          1 year ago

          The rclone mount works via SSH credentials. Torrent files and tracker searches run over simple HTTPS, since both my torrent client and jackett expose public APIs for these purposes, so I can just enter the web address of these endpoints into the apps running on my homelab.

          Sidenote, since you said sshfs mount: I tried sshfs, but has significantly lower copy speeds than with rclone mount. Might have been a misconfiguration, but it was more time efficient to use rclone than trying to debug my sshfs connection speed.

          • @notfromhere@lemmy.ml
            link
            fedilink
            English
            21 year ago

            I have noticed very slow speeds with sshfs as well. I’ll have to give rclone mount over ssh a try. Thanks!

  • @Fedegenerate@lemmynsfw.com
    link
    fedilink
    English
    51 year ago

    I am sorry, I am but a worm just starting Docker and I have two questions.

    Say I set up pihole in a container. Then say I use Pihole’s web UI to change a setting, like setting the web UI to the midnight theme.

    Do changes persist when the container updates?

    I am under the impression that a container updating is the old one being deleted and a fresh install taking its place. So all the changes in settings vanish.

    I understand that I am supposed to write files to define parameters of the install. How am I supposed to know what to write to define the changes I want?

    Sorry to hijack, the question doesn’t seem big enough for its own post.

    • @Limit@lemm.ee
      link
      fedilink
      English
      61 year ago

      With containers, most will have a persistent volume that is mapped to the host filesystem. This is where your config data is. When you update a container, just the image is updated(pihole binaries) but it leaves the config files there. Things like your block lists and custom dns settings, theme settings, all of that will remain.

  • @thorbot@lemmy.world
    link
    fedilink
    English
    21 year ago

    I just have a UniFi firewall, a Synology Diskstation, and a linux server running everything. Provides torrenting, video streaming with plex, file sharing, game server hosting, music hosting, and more, and I don’t ever have to mess with it :). This is impressive but I don’t know if I would want to support it personally

    • @7Sea_Sailor@lemmy.dbzer0.comOP
      link
      fedilink
      English
      21 year ago

      I’d love to have everything centralized at home, but my net connection tends to fail a lot and I dont want critical services (AdGuard, Vaultwarden and a bunch of others that arent listed) to be running off of flakey internet, so those will remain in a datacenter. Other stuff might move around, or maybe not. Only time will tell, I’m still at the beginning of my journey after all!

      • @thorbot@lemmy.world
        link
        fedilink
        English
        11 year ago

        Fair. I’m lucky enough to be able to get business internet at home so I have a static IP and 99.9% uptime. My plex watchers and game hosting players know that sometimes around 3am, they might be booted when my networking gear auto updates itself, haha

  • Lunch
    link
    fedilink
    English
    11 year ago

    Since nobody else asked about this, why ruTorrent over the other typical download clients?

      • Lunch
        link
        fedilink
        English
        11 year ago

        Ahh I’m not too familiar with seedboxes, thanks 🌻

    • @7Sea_Sailor@lemmy.dbzer0.comOP
      link
      fedilink
      English
      61 year ago

      Pretty sure ruTorrent is a typical download client. The real reason is that it came preinstalled and I never had a reason to change it ¯_(ツ)_/¯

  • @remotedev@lemmy.ca
    link
    fedilink
    English
    31 year ago

    Sorry if someone already asked this, but do you have any tutorials or guides that you used and found helpful for starting out? I have some small experience with nginx and such, but I would definitely need to follow along with something that tells me what to do and what each part does in a infrastructure like you have haha