Possibly linux@lemmy.zip to LocalLLaMA@sh.itjust.worksEnglish · edit-25 days agoAm I the only one who is really impressed by Granite4 from IBM?message-squaremessage-square6fedilinkarrow-up110arrow-down10file-text
arrow-up110arrow-down1message-squareAm I the only one who is really impressed by Granite4 from IBM?Possibly linux@lemmy.zip to LocalLLaMA@sh.itjust.worksEnglish · edit-25 days agomessage-square6fedilinkfile-text
minus-squareXylight@lemdro.idlinkfedilinkEnglisharrow-up3·edit-24 days agothere’s also a “small” and “micro” variant, which are 32b a6b MoE and 3b dense models respectively
minus-squareBaŝto@discuss.tchncs.delinkfedilinkEnglisharrow-up1·3 days agogranite4:micro-h should be able to run on machines with 4GB RAM
minus-squareXylight@lemdro.idlinkfedilinkEnglisharrow-up2·3 days agoYou can run Qwen3 4b thinking at q4 quantization at 2.5GB, which is probably a better model too
there’s also a “small” and “micro” variant, which are 32b a6b MoE and 3b dense models respectively
granite4:micro-h should be able to run on machines with 4GB RAM
You can run Qwen3 4b thinking at q4 quantization at 2.5GB, which is probably a better model too