TL;DR:
- I can’t decide between Debian and the new “immutable” Fedora server variants
- Currently I use Debian with pretty much everything being containerised, and it works fine.
- I’m neither very good at what I’m doing, nor want to spend my weekends troubleshooting. Opting for something new could cause some headaches I guess?
- How did you set up CoreOS? Are there simple ways?
- Would you recommend me something different?
My backstory with Debian
I will soon set up a new home server and need your opinion and experiences.
I’m using Debian as the OS for my current one.
While it doesn’t match my “taste” perfectly, as I slightly prefer RedHat stuff, I really don’t have much preference, since I don’t interact with the host much anyway.
Everything is containerised via Docker, and I don’t even know why I like Rocky-/ Alma more. I tried Alma once and it just clicked better, I can’t explain it…
But that doesn’t mean I dislike Debian, not at all!
Still, at that time I decided to go with Debian, since it’s the standard for most selfhosters, has the best software support, and is completely community run, opposed to RHEL and its clones.
At that time I didn’t know Distrobox/ Toolbx, and I really wanted to install CasaOS (basically a simplified Cockpit + Portainer for less techy people), because I was a total noob back than and didn’t want to do everything via CLI.
Nowadays, I found alternatives, like Cockpit, and I also do more via the terminal.
And if I want to install something that doesn’t support my host OS, then I just enter my Toolbx and install it there.
Still, I absolutely don’t regret going for Debian. It was a good choice. It’s solid and doesn’t get in my way.
What has changed in the last year(s)
In the last year now, I really began to enjoy using image based distros, especially Fedora Atomic.
I really love Atomic as desktop distro, because it is pretty close to upstream, while still being stable (as in how often things change).
For a desktop workstation, that’s great, because DEs for example get only better with each update imo, and I want to be as close to upstream as possible, without sacrificing reliability, like on a rolling release.
The two major releases each year cycle is great for that.
But for a server, even with the more stable kernel that’s used in CoreOS from what I’ve heard, I think that’s maybe too unstable?
I think Debian is less maintenance, because it doesn’t change as often, and also doesn’t require rebooting after each transaction.
But, on the contrary, I wouldn’t loose much to the “immutability”, because I use containers for everything anyway.
Having way better security (sane SELinux setup, rootless containers, untampered OSA, etc.) and the ability to roll back in case something doesn’t work, while self updating, sounds very promising.
Setting up CoreOS; FCOS vs FIOT
The major thing that’s keeping me away from CoreOS/ uCore is all the ignition-butane-stuff.
From what I’ve heard, it’s needlessly complicated for home use, and FCOS is best suited for fleets/ clusters of servers, not just for one.
Fedora IOT seems to be simpler, but doesn’t have the same great defaults and features as uCore, since there isn’t an IOT variant of uBlue.
But hey, at least I have my Anaconda installer.
What do you think about installing IOT, and then rebasing to uCore?
Or, do you think FCOS is just not the right thing for my use case?
In general, do you think that it is worth it, compared to plain old Debian?
Pros vs. cons
Anyway. I’m really thinking about all of this for a long time now, and can’t decide.
On the one side, it all sounds promising and great.
But, on the other side, selfhosting isn’t a primary hobby of mine.
I just want a solid setup I don’t have to maintain much after setting everything up. Image based server OSs are still very new and often unheard of, and being an early adopter might cause a lot of headache in that case when it comes to servers.
The “right” use case?
Just in case no one has tried FCOS or FIOT here, I will continue using Debian for my main server, and only use Fedora IOT for my Octoprint server, which only gets turned on sporadically, and would greatly benefit from that.
But if there are positive experiences, then I might give it a try.
Alternatives
Or, would you recommend me something entirely different?
NixOS for example sounds great in theory, but is way too complicated for me personally.
Or, would you recommend me to give Alma another try?
Is there something even better?
I had problems with podman/selinux and jellyfin and gpu acvelleration which is why I’m on debian now. I’d go with atomic any day if I could solve the problem but I don’t know how.
I’ve been using Fedora CoreOS as my main server in my home lab for a couple years now and have been very happy with it. You still get the convenience of automatic atomic updates and you also have the flexibility of being able to install whatever additional Fedora packages you need with rpm-ostree.
I installed Docker Engine and docker-compose from upstream. You should be able to set SELinux to permissive mode if you need to so it doesn’t enforce any policies.
Setting SELinux to permissive is not a good security practice
You’re right and I’m not saying that I recommend settling it to permissive. You should understand the risks involved.
Do you need to reboot after every update or updates applied live? Only have experience with rpm-ostree on kinoite
You do. It automatically reboots after applying rpm-ostree updates.
deleted by creator
Thx!
I wasn’t able to debug it. Maybe next year or so
I migrated my home- and webservers from Debian to FCOS a while ago and I’m very happy with how everything works.
Troubleshooting butane/ignition was a bit of a pain in the butt but worth it imo. I suggest just reading through the FCOS docs, they guided me well while setting everything up. I use podman on my webservers and docker on the homeserver (bc nextcloud aio is not fully podman compatible). I use the installer to build a pre-configured ISO that I can deploy where I want to.
Someone in the comments mentioned Flatcar, which I think looks compelling as well, since it’s basically the same but more of a community effort.
I use UCore for my homelab and it’s been flawless. Absolutely no issues. I run around 50 containers, LLMs, and host some public sites with Caddy.
The major thing that’s keeping me away from CoreOS/ uCore is all the ignition-butane-stuff. From what I’ve heard, it’s needlessly complicated
It’s super super easy. Run a docker one-liner on your existing local server or laptop to host a quick webserver:
docker run -p 5080:80 --name quick-webserver -v "$PWD":/var/www/html php:7.2-apacheAnd put this Ignition file in the directory from above: https://github.com/ublue-os/ucore/blob/main/examples/ucore-autorebase.butane
That’s it, that’s the only steps. Boot off the ISO and type in the hosting URL from above.
You’ll only need that when building the server the first time.
I started using Fedora Silverblue on a tablet, seems to work fine so far, but requiring a reboot in order to install new system packages is a bit cumbersome and the process itself takes a while, but ordinary Fedora also doesn’t win any races when asked to install a new package
I think switching to FCOS or Flatcar on servers that just use containers makes sense. Since it lessens the burden of administrating the base system itself. Using butan/ignition might be unusual at first, but it also allows to put the base system configuration into a git repo, and makes initial provisioning using ansible or similar unnecessary. The rest of the system and services can be managed via portainer or similar software.
I also do not have long term experience with FCOS, but the advertised features of auto-update, rolling-release, focus on security and stability makes it a good fit for container servers, IMO.
An alternative to Debian on servers might also be Apline Linux. Which also has more a focus on network devices, but some people use it on a desktop as well.
If you have many different systems, and just want to learn to operate them all, maybe NixOS might be interesting. Using flakes, you can configure multiple machines from just one repo, and share configurations between them. But getting up to speed on NixOS might not be so easy, it has a steep learning curve.
I used CoreOS for awhile but moved to flatcar, same stack better support.
I am not a fan of the ublue distros. Add a lot of complication, and causes me problems.
