• 0 Posts
  • 34 Comments
Joined 1 year ago
cake
Cake day: July 23rd, 2023

help-circle







  • I’ve been testing Ollama in Docker/WSL with the idea that if I like it I’ll eventually move my GPU into my home server and get an upgrade for my gaming pc. When you run a model it has to load the whole thing into VRAM. I use the 8gb models so it takes 20-40 seconds to load the model and then each response is really fast after that and the GPU hit is pretty small. After I think five minutes by default it will unload the model to free up VRAM.

    Basically this means that you either need to wait a bit for the model to warm up or you need to extend that timeout so that it stays warm longer. That means that I cannot really use my GPU for anything else while the LLM is loaded.

    I haven’t tracked power usage, but besides the VRAM requirements it doesn’t seem too intensive on resources, but maybe I just haven’t done anything complex enough yet.





  • For 99% of people an online password manager like Bitwarden or LastPass is going to significantly help them manage passwords securely despite the risks associated with cloud services. Most people can’t handle self hosting Bitwarden or syncing a Keepass database by themselves. Without an easy to access and easy to use online option people will revert to significantly riskier methods like password reuse or using some sort of repeatable/guessable pattern.

    For the 1% of people who want more security there are options like Vaultwarden or Keepass. Even then it’s not uncommon to make mistakes and lose data/access or leave some sort of vulnerability exposed. The attack surface is a lot smaller than a public service though which is beneficial.