k3s is brilliant. we run production clusters on it.
The problem with k3s is that the architecture level libraries are a bit outdated. Early on, it was for a particular reason - ARM64 (raspberry pi) support.
But today, like everyone is on ARM - even AWS.
For example the network library is Flannel. Almost everyone switches to Calico for any real work stuff on k3s. it is not even a packaged alternative. Go-do-it-urself.
The biggest reason for this is a core tenet of k3s - small size. k0s has taken the opposite approach here. 50mb vs 150mb is not really significant. But it opens up alternative paths which k3s is not willing to take.
In the long run, while I love k3s to bits....I feel that k0s with its size-is-not-the-only thing approach is far more pragmatic and open for adoption.
Agreed on 100% of your points -- you've hit on some of the reasons I chose (and still choose) k0s -- Flannel is awesome but it's a little too basic (my very first cluster was the venerable Flannel setup, I've also done some Canal). I found that k0s's choice of Calico is the best -- I used to use kube-router (it was and still is amazing, great all-in-one tool) heavily but some really awesome benchmarking work[0] caused me to go with Calico.
Most of the other choices that k0s makes are also right up my alley as well. I personally like that they're not trying to ride this IoT/Edge wave. Nothing wrong with those use cases but I want to run k8s on powerful servers in the cloud, and I just want something that does it's best to get out of my way (and of course, k0s goes above and beyond on that front).
> The biggest reason for this is a core tenet of k3s - small size. k0s has taken the opposite approach here. 50mb vs 150mb is not really significant. But it opens up alternative paths which k3s is not willing to take.
Yup! 150MB is nothing to me -- I waste more space in wasted docker container layers, and since they don't particularly aim for IoT or edge so it's perfect for me.
k3s is great (alexellis is awesome), k0s is great (the team at mirantis is awesome) -- we're spoiled for choice these days.
Almost criminal how easy it is to get started with k8s (and with a relatively decent standards compliant setup at that!), almost makes me feel like all the time I spent standing up, blowing up, and recreating clusters was wasted! Though I do wonder if newcomers these days get enough exposure to things going wrong at the lower layers as I did though.
actually k3s has a cloud deployment startup - civocloud also using it. I would say that the production usage of k3s is outstripping the raspberry pi usage...but however the philosophical underpinnings remain very rpi3 centric.
Things like proxy protocol support (which is pretty critical behind cloud loadbalancers), network plugin choice, etc is going to be very critical.
The problem with k3s is that the architecture level libraries are a bit outdated. Early on, it was for a particular reason - ARM64 (raspberry pi) support. But today, like everyone is on ARM - even AWS.
For example the network library is Flannel. Almost everyone switches to Calico for any real work stuff on k3s. it is not even a packaged alternative. Go-do-it-urself.
The biggest reason for this is a core tenet of k3s - small size. k0s has taken the opposite approach here. 50mb vs 150mb is not really significant. But it opens up alternative paths which k3s is not willing to take.
In the long run, while I love k3s to bits....I feel that k0s with its size-is-not-the-only thing approach is far more pragmatic and open for adoption.