A simpler answer might be llamafile if you’re using Mac or Linux.
If you’re on windows you’re limited to some smaller LLMs without some work. In my experience the smaller LLMs are still pretty good as chat bots so they might translate well.
A simpler answer might be llamafile if you’re using Mac or Linux.
If you’re on windows you’re limited to some smaller LLMs without some work. In my experience the smaller LLMs are still pretty good as chat bots so they might translate well.
Llamafile is a great way to get use an LLM locally. Inference is incredibly fast on my ARM macbook and rtx 4060ti, its okay on my Intel laptop running Ubuntu.