

Phew, to run such a model - even just 27B quantized - at a reasonable speed requires some serious GPU Power.
Best thing I have is a 7800 XT 16gb (so ROCm it would be), don’t think that’s realistic and I’d rather invest the necessary amount of money for new GPUs into sth. more useful like a small electric vehicle than a slop cannon 🥴. Happy to hear actual useful things can be retained after the bubble finally crashes though. Oh please, let it crash tomorrow… together with the housing market…


lol
??
Is this whole text LLM generated and you just copy-pasted everything including the (now unlinked) citations and the orders the machine added to the text without double-checking?