mariah@feddit.rocks to Free Open-Source Artificial Intelligence@lemmy.worldEnglish · edit-26 months agoIs there a good ai thats run fast on a amd rx 7600?message-squaremessage-square9fedilinkarrow-up111arrow-down12file-text
arrow-up19arrow-down1message-squareIs there a good ai thats run fast on a amd rx 7600?mariah@feddit.rocks to Free Open-Source Artificial Intelligence@lemmy.worldEnglish · edit-26 months agomessage-square9fedilinkfile-text
Ive been playing koboldai horde but the queue annoys me. I want a nsfw ai for playing on tavernai chat
minus-squareprojectmoon@lemm.eelinkfedilinkEnglisharrow-up1·5 months agoHow much speed are you actually getting on Mixtral (I assume that’s the 8x7b). I have 64 GB of RAM and an AMD RX 6800 XT with 16 GB of VRAM. I get like 4 tokens per second with Q5_K_M quant.
How much speed are you actually getting on Mixtral (I assume that’s the 8x7b). I have 64 GB of RAM and an AMD RX 6800 XT with 16 GB of VRAM. I get like 4 tokens per second with Q5_K_M quant.