ProperlyProperTea, 10 months ago

I have an RX6800XT and I use KoboldCPP to run models I download off of Huggingface.

I’m not sure how many tokens per second it generates, probably about 10?

If you want to try it yourself here’s a link to the Github page: github.com/LostRuins/koboldcpp