• 0 Posts
  • 4 Comments
Joined 1 year ago
cake
Cake day: November 15th, 2023

help-circle

  • If you are just starting and messing around you can go a long way with a single node k3s cluster (I prefer nixos since it makes managing and replicating things REALLY repeatable, but it is it’s own rabbit hole)

    BUT if you need several 9s your going to need even more than just 1 server with k3s on it. Your gonna want redundancy, monitoring, and processes.

    1. 3 nodes while only using capacity of 2
    2. Shared volume infra like ceph or a nas
    3. Load balancing firewall like opnsense
    4. Multiplexed internet
    5. UPS for power issues
    6. Onsite backups + cloud backups
    7. Kube-prometheus-stack (or the contents of)
    8. KEDA (for auto scaling)

    (Not a day 0 recommendation)

    The reason kubernetes is complex (and hard to learn) is cause it kinda forces you to consider all kinds of reliability, and scaling issues, that you may not need for a while.

    If you only have one machine, it does feel like a bit much to NEED an autoscaler.

    You can create a vanilla cron job that runs a docker container command so you don’t have to “install” anything on your node. L

    You can use multiple docker compose files to manage stuff independently so you can upgrade stuff without affecting other things.

    I know you say you want auto scaling, but what are you autoscaling against? Like is something else scaling up at different intervals? I think a thing to question is if your extra instances ever need to scale down. Auto scaling is a cost saving measure and if you have static infrastructure with no other load then why ever scale down? Do your cron jobs take too many resources and you have to scale down your micro services? If so you’ve got way more to consider that just plain autoscaling, and maybe you need to scale your infrastructure in which case your back to questioning whether or not you need to scale down.

    I’m questioning your requirements only because if you are trying to just “get something done” k8s and nomad are going to be a distraction since you aren’t already familiar with them. If learning k8s or nomad is also part of your goal, then awesome, I would definitely suggest k3s.