mapumbaa@lemmy.zip to LocalLLaMA@sh.itjust.worksEnglish · 7 days agoGPT-OSS 20B and 120B Models on AMD Ryzen AI Processorswww.amd.comexternal-linkmessage-square12fedilinkarrow-up116arrow-down10
arrow-up116arrow-down1external-linkGPT-OSS 20B and 120B Models on AMD Ryzen AI Processorswww.amd.commapumbaa@lemmy.zip to LocalLLaMA@sh.itjust.worksEnglish · 7 days agomessage-square12fedilink
minus-squareKissaki@programming.devlinkfedilinkEnglisharrow-up2·edit-26 days agoFor those interested in the desktop-capable requirements: 16 GB GPU. 9070 XT? For lighting fast performance with the OpenAI GPT-OSS 20B model, users can use the AMD Radeon™ 9070 XT 16GB graphics card in a desktop system. Does it require that gen?
minus-squareafk_strats@lemmy.worldlinkfedilinkEnglisharrow-up1·1 day agoIf your video card has 16+ Gb of memory you will be able to run it with: NVIDIA cards GTX 10 series or later on ollama. Ollama is easy, but it leaves a lot of performance on the table. If you have less than 16 GB, you may be able to get good performance using llama.cpp or especially Ik_llama.cpp.
minus-squareJayjader@jlai.lulinkfedilinkEnglisharrow-up2·6 days agoNot sure it requires that gen, but it definitely needs the 16GB. I have a 6700 (XT?) with 12 GB and ollama loads up gpt-oss on my CPU. Whereas models like deepseek-r1 with 14b parameters runs solely on the GPU.
For those interested in the desktop-capable requirements: 16 GB GPU. 9070 XT?
Does it require that gen?
If your video card has 16+ Gb of memory you will be able to run it with: NVIDIA cards GTX 10 series or later on ollama.
Ollama is easy, but it leaves a lot of performance on the table.
If you have less than 16 GB, you may be able to get good performance using llama.cpp or especially Ik_llama.cpp.
Not sure it requires that gen, but it definitely needs the 16GB. I have a 6700 (XT?) with 12 GB and ollama loads up gpt-oss on my CPU. Whereas models like deepseek-r1 with 14b parameters runs solely on the GPU.