Xylight@lemdro.id to LocalLLaMA@sh.itjust.worksEnglish · edit-22 months agoMy 8gb vram system as i try to load GLM-4.6-Q0.00001_XXXS.gguf:media1.tenor.comexternal-linkmessage-square13fedilinkarrow-up191arrow-down15
arrow-up186arrow-down1external-linkMy 8gb vram system as i try to load GLM-4.6-Q0.00001_XXXS.gguf:media1.tenor.comXylight@lemdro.id to LocalLLaMA@sh.itjust.worksEnglish · edit-22 months agomessage-square13fedilink
minus-squareffhein@lemmy.worldlinkfedilinkEnglisharrow-up1·1 month agoAh, multiple GPUs? For some reason I thought you meant that with exllamav3 you had managed to load a model which was larger than your VRAM.
Ah, multiple GPUs? For some reason I thought you meant that with exllamav3 you had managed to load a model which was larger than your VRAM.