More luck than anything really. It was probably because it had 6 months left and the fact that reading and writing felt slow. Everything else behaved normally and buying a new disk was an educated guess that turned out to be the correct choice.

I had a weird issue with a server SSD.

6 months ahead of scheduled swap, it didn’t die, it just started reading and writing really sluggishly, making the whole server behave really weird. Disk smart statistics looked healthy and disk self tests passed with flying colors. Anyway, had to swap it early and do a re-install of the OS.

The rest of my cluster temporarily took over running some pods and only saw downtime for a few pods that were dependent on some disks in the failing server.

I guess the incident has restarted my interest in distributed storage.

Man, I know that feeling. One thing that helped me better deal with issues like this, was to have a changelog. Basically I write down what a setting was, what I changed it to and a reason. If something goes wrong, I can at least undo what changes I’ve made and see if it helps. It’s not perfect, but it might shave some hours off a RCA.

Absolutely agree. I have been thinking of starting a selfhost guide that takes you through the different ways to selfhost and the basic concepts of it, but gave up because I’m a shit writer and my experiences are mostly docker, k8s and Terraform/OpenTofu.

This is the same strategy I use with my family as well, I refer to a service they know and then tell them it’s in our house.

Currently rewriting my homelab into terraform and adding some redundancies using cloud environments, in case of power outages or network issues.

Currently trying to figure out how to create and maintain an internal CA in order to enable pod to pod TLS communication, while using letsencrypt for my public ingresses.

Excellent point, thank you!

I use dperson/samba, which is both simple and comes with usage examples.

I will be moving my entire homelab to a different country, which currently consist of two kubernetes nodes, a NAS and various home automation devices. I will be scaling down gradually, taking cold storage backups of everything and plan to resurrect everything on new hardware once I have moved.

I subscribe to the selfh.st newsletter

I used to use ansible and helm, but it is overkill for my case. Today I basically use a combo of markdown and bash scripts, the combination of them allows me to run the scripts straight from my IDE.

I had the same idea a couple years back and even though I would love something that you download and just run and it would work, I realized that in order to get a decent adoption rate, you would need a whole ecosystem, similar to apple in order for it to work.

I still think you can develop something like a hub where you install services like apps, but I doubt it would attract anyone outside selfhosting circles.

There are several things you can and should do to harden your server, many of them can be found here.

Wouldn’t tar --help suffice? Afaik, it returns exit code 0.

Welcome to projects 101 where you are now in charge of coding, infrastructure, logging, metrics, secrets, linting and deployment.

You could look into apps like authelia, keycloak, authentic, etc.

I run my self hosted stuff on a k3s cluster at home on bare metal, then use cloudflare to protect the IP and access only by VPN.

Good suggestion. I am a bit averse to bitnami unfortunately but it might beat writing all the charts myself.

Good suggestion, but I find it hard to find the source of those charts (might be a mobile thing though). I will be sure to check it out more closely on my desktop.