In the case of Machine learning the term has sort of been morphed to mean “open weights” as well as open inference/training code. To me the OSI is just being elitist and gate keeping the term.
In the case of Machine learning the term has sort of been morphed to mean “open weights” as well as open inference/training code. To me the OSI is just being elitist and gate keeping the term.
Koboldcpp should allow you to run much larger models with a little bit of ram offloading. There’s a fork that supports rocm for AMD cards: https://github.com/YellowRoseCx/koboldcpp-rocm
Make sure to use quantized models for the best performace, q4k_M being the standard.