I’ve been using the bitwarden android app for about a year and a half with zero issue 🤷

Yes, llama-swap and I use it for home assistant text-gen notifications, basic coding tasks, etc

If anyone here self-hosts definitely check out llama-swap as it has some nifty features for hotswapping LLMs, image generation models and voice models.

Bruno

Extremely conservative/risk averse and slow to implement basically anything new, unless it’s super flashy like AI lol

This is really cool in theory, I doubt my corp would let me use it though :(

Oh definitely, it was overwhelming at first for me too, but that’s how one learns imo

Now that I’ve been using it for a year and a half it’s much better but those first few months were quite the learning experience lol

Surprised no one has mentioned proxmox (at least not the top 10 threads I saw first)

Basically debian with a webui specifically for spinning up VMs and LXCs and managing storage and such, then install whatever distro in an LXC and run docker in that.

Depends on how much quantization, but still fairly beefy, couldn’t run it on my homelab with a 3080ti for example.

I generally use smaller 8-12b models and they’re alright depending on the task.

Proxmox as another option

Does proxmox count? Then I run lots of docker containers in lxcs

Vaultwarden

There’s a silly bug where if you use Ventoy to install proxmox it fucks up the efi partition and won’t allow boot without the USB in, I left that USB in for probably 6 months before bothering to fix it lol.

There’s no arguing with anti-AI people it’s pointless

“Having common functionality around certain types of fediverse content is centralization.”

Maybe consider moving to an instance that includes basic functionality?

74 across 2 proxmox nodes in a few lxcs

Authelia does have it too, I use that with traefik

This but just JF instead of Navidrome

^Prompt Engineer

Its a leap to say “nvidia AI support on Linux is bad” when you mean a very particular set of circumstances (which don’t apply to someone who would just be getting into it as they’re using consumer grade hardware) causes issues.

I have a 3080ti and run 8b - 12b all in VRAM just fine, which is what a majority of people getting into it would be doing as well, again to my point about pulling an 18 wheeler trailer with a horse, you’ve got worse problems then nvidia on Linux (if you’re trying to run a 70b model on consumer hardware).