So, I’m trying to roll a K3s cluster, basically to learn new stuff (yup, I definitely don’t need it, but I’d like to).
I imagined it to be composed by a master plane (virtualized in a server in my house) and a node worker (on a Hetzner VPS). The worker node should, for the moment, only serve a reverse proxy to route connections to my home server (which is gonna host Synapse, Mastodon and some things, perhaps) and later on maybe SSO and other thingies.
This is my problem: I guess the best way to route the traffic between the w.n. and the m.p. would be a Wireguard tunnel, which, I suppose, I could directly manage via kubectl. You see where this is going. Until I a have a working tunnel I cannot manage the remote working node, but at the same time I need a tunnel to be able to connect to said working node.
So, I guess my idea has some flaws. How could I manage to do it?
Make sure you read up on latency requirements for k8s too. It’s definitely doable but etcd for example needs really low latency between nodes to not cause issues by default.
If you only have one master like you described, it’d probably be fine.
I’d considered doing something similar at some point but couldn’t quite figure out what the likely behaviour was if the workers lost connection back to the control plane. I guess containers keep running, but does kubelet restart failed containers without a controller to tell it to do so? Obviously connections to pods on other machines will fail if there is no connectivity between machines, but I’m also guessing connections between pods on the same machine will be an issue if the machine can’t reach coredns?
I’ve done it with docker swarm and it was awful, the connection latency would break the cluster constantly
Interesting. I didn’t have time to test it yet, anyway