But the estimation is with each NC instance with half a CPU and 1GB of memory. This is a super conservative estimation, that doesn’t include anything besides a tiny Fargate deployment and Aurora instances.

Edit: fargate ($40/month), the tiniest Aurora instances at 20% utilization and with merely 50GB storage ($120/month). Missing s3, which will easily cost $50 in storage and transfer (for only a few TB), ALBs and network traffic, especially outbound (easily $50-100 depending on volumes).

This basic solution’s real cost is already between $150 and $300/month. I don’t know NC enough to understand volumes on DBs and all usage, but I assume that it’s going to be lots of data in and out (backups, media, etc.). —edit—

For a heavily used NC instance (assuming a company offering it as a service), the cost is going to become massive pretty fast.

Also, as I side note, if a company is offering NC as a service, but doesn’t manage a single piece of NC deployment… What is the company product? And most importantly, how are they going to make money when AWS is going to eat a linearly scalable chunk of their revenue forever?

Well yeah, wouldn’t break the bank, but a conservative cost estimate (without considering network costs, for example, quite relevant for a data intensive app) would bring this setup to about $40/month. That is about 5 times more expensive than a VPC with 4x the resources.

OP said this is some sort of “enterprise self-hosting” solution, which I guess then kind of makes sense. For a company providing nextcloud as a service I would never vendor lock myself and let AWS take a huge chunk of my revenue forever, but I can imagine folks have different opinions.

Everyone is free to pick their poison, but I have to ask…why? What is the target audience here? This is a massively overkill architecture IMHO. Not to talk about the fact you now need 3 managed services (fargate, s3 and aurora at least) for a single self hosted tool, and that is being generous (not counting cloudwatch, ALBs, etc.).

• Why do you need security groups to allow egress anywhere (or, at all)?
• I would pin the image to a digest, rather than using latest.
• what is the average monthly cost for this infra for you?

Oh, it looks like! Something went wrong with Zola build and I must have not noticed. Thanks a lot for pinging me about that, I will fix it today!

EDIT: Fixed! That’s what you get when you forget to bump the Docker image version after you upgrade zola version locally with a breaking change in the config! :) Thanks for letting me know, it would have taken me a long time to see it was broken!

Hey, the short answer is yes, you can.

I would elaborate a little more:

• First, you have the problem of sourcing the data. In essence, Crowdsec won’t be able to go and fetch those logs for you dynamically, but can go and take those logs from a file (you can do a dirty solution like a sidecar deployment) or from a stream. You can deploy crowdsec in multiple modes, and you can have many instances that talk to each other. You can also simply have some process tailing the pod logs and sending them to a file crowdsec has access to or serving them as a stream (see https://doc.crowdsec.net/docs/data_sources/intro).
• The above means that it doesn’t really matter whether you run Crowdsec inside your cluster (it does have a Helm chart) or on the host. Ultimately all it matters is that crowdsec has access to your pods logs (for example, the logs of your ingress controller).
• The next piece is the remediation component. What do you want crowdsec to do, once it is able to detect bad IPs? If you want to just add IPs to the firewall, then it might make more sense running it on the host(s) you use in ingress, if you want to add the IPs to network policies you can do it, but you need to develop your own remediation components. I am planning to write a remediation component that will add the IPs to Hetzner firewall, some other systems are already supported, but this would be a way to basically block the IPs outside your cluster. For nginx ingress controller there is already a pre-made remediation component .

In practice I personally would choose a simple setup where the interesting logs are just forwarded (in Syslog format for example) to a single crowdsec instance. If you have ingress from a single node, I’d go for running it on the host and banning via firewall, if you have multiple ingress nodes, then I would run it inside the cluster and ban via a loadBalancer/cloud firewall/whatever you have in front.

In essence, I would spend some time to think about your preferences, and it might take a little bit to make the setup clean, but I think you have plenty of flexibility to do what you prefer. Let me know if you want to bounce some more ideas!

Yes, I have used it in the past and it was annoying…

You can get SSL certs with letsencrypt, but you need to use the http verification method.

Completely agree, which is why I do the same.

Additional bonus: proxies that interact with the docker API directly (I think also caddy can do it) save you from exposing the services on any port at all (only in the docker network). So it’s way less likely to expose a port with a service by mistake and no need for arbitrary and unique localhost ports.

Since you run already OpenWrt, you can check out https://openwrt.org/docs/guide-user/services/ddns/client

There is a list on this page of compatible services. If you don’t want to use one more service (DNS), you can use a domain registrar with an API (like porkbun) and find online tools that work with that.

Be aware of the risks of hosting your websites publicly from home, make sure to run them in very isolated environments. Having your VPS compromised is bad, but having your home network compromised is much worse!

That is basically the essence of this post too! Except crowdsec is used to do what fail2ban does + some light form of WAF (without spinning another machine - which is not strictly needed for a WAF, you can use owasp modsecurity-ready proxies).

Thanks! I did mention this briefly, although I belong to the school that “since I am anyway banning IPs that fail authentication a few times, it’s not worth changing the port”. I think that it’s a valid thing especially if you ingest logs somewhere, but if you do don’t choose 2222! I have added a link to shodan in the post, which shows that almost everybody who changes port, changes to 2222!

Yes, pretty much that. Plus some configuration might be easier with a DNS hosting. But the main benefit is decoupling domain and DNS for easier change.

Been there…

I thought my API keys were expired, I regenerated them, changed a couple of things, checked all API calls to see if they changed API itself…then I searched the exact error and found out.

For such a breaking change to the API, was it hard to drop an email to every account not meeting the damn “requirements” with an API call performed in the last x months, to alert of the change?

I also use porkbun, their API is not a masterpiece but it works and allows you to get, set and update records. In fact their API is now supported by some of the common ddns scripts out there.

That’s a very interesting gotcha. They don’t seem to support address ranges either. Unless once you add the whitelist the requests still work from any address (their documentation is ambiguous). This is even more confusing.

I also migrated everything to Porkbun. Gandi used to be good too, we used it extensively at work in my previous org (~3 years ago).

Is the whole sector regressing? It seems these companies aren’t happy just earning a profit based on the service they offer. There is always something “more” that they need to do. Often this makes the experience worse. Meh.

Super happy with Porkbun BTW, it just works, does what it’s needed and I found the renewals to be 50% cheaper compared to GoDaddy…

I found it on their FAQ.

Yes, it is generally less restrictive, but… I have 4 domains, and now I have renewed all of them for the maximum amount. They will all expire after 2033. So unless I decide to add more domains (which is unlikely), I won’t spend a cent in the next ~9 years. I wonder if they really enforce it as it is written or they consider still the renewal an expense “split” over the duration.

Still, I really don’t understand. You can - and should - have proper rate limits on the API. You have API keys that uniquely identify the source, what is “the abuse” they are trying to prevent this way…?

NameCheap

WOW! I did not know that. I just checked and after a little search:

We have certain requirements for activation to prevent system abuse. In order to have API enabled, your account should meet one of the following requirements:

- have at least 20 domains under your account;
- have at least $50 on your account balance;
- have at least $50 spent within the last 2 years

$50 in last 2 years is not much, but for those who renew for many years, it is still stupid.

Ironically, Namecheap is what the people in https://github.com/navilg/godaddy-ddns/issues/32 migrated to!

I really wish that domain registration was done in a different way, but even in current scenario, gutting features for such a basic service to extract a few bucks and risking losing customers…?

ClouDNS

I think I heard of it. I think most DDNS scripts support a lot of registrars as well, if one doesn’t want to go with full DNS hosting.

In case of DNS hosting (I also linked it in the post, but it’s a good shotout), there is desec.io too. EU-hosted, free (although donations are highly encouraged) and has a tons of features! There is also a Terraform provider!

I run Prometheus on a separate cluster, so I plug my servers with node_exporter and scrape metrics. I then alert with grafana. To be honest, the setup is heavier (resource usage-wise) than I would like for my use case, but it’s what I am used to, and scales well to multiple machines.