🇦🇺𝕄𝕦𝕟𝕥𝕖𝕕𝕔𝕣𝕠𝕔𝕠𝕕𝕚𝕝𝕖@lemm.ee to LocalLLaMA@sh.itjust.worksEnglish · 1 year agoHow much gpu do i need to run a 90b modelmessage-squaremessage-square16fedilinkarrow-up113arrow-down11file-text
arrow-up112arrow-down1message-squareHow much gpu do i need to run a 90b model🇦🇺𝕄𝕦𝕟𝕥𝕖𝕕𝕔𝕣𝕠𝕔𝕠𝕕𝕚𝕝𝕖@lemm.ee to LocalLLaMA@sh.itjust.worksEnglish · 1 year agomessage-square16fedilinkfile-text
minus-squarered@lemmy.ziplinkfedilinkEnglisharrow-up1·1 year agothis is useless, llama.cpp already does that airllm does (offloading to CPU) but its actually faster. so just use ollama
this is useless, llama.cpp already does that airllm does (offloading to CPU) but its actually faster. so just use ollama