When you’re talking about the PCIe peripheral, you’re talking about a separate dedicated graphics card or something else?
Yes, similar to what a PCIe Graphics Card does.
A PCIe slot is the slot in a desktop motherboard that lets you fit various things like networking (ethernet, Wi-Fi and even RTC specialised stuff) cards, sound cards, graphics cards, SATA/SAS adapters, USB adapters and all other kinds of stuff.
I guess the main point of NPUs are that they are tiny and built in
GPUs are also available built-in. Some of them are even tiny.
Go 11-12 years back in time and you’ll see video processing units embedded into the Motherboard, instead of in the CPU package.
Eventually some people will want more powerful NPUs with better suited RAM for neural workloads (GPUs have their own type of RAM too), not care about the NPU in the CPU package and will feel like they are uselessly paying for it. Others will not require an NPU and will feel like they are uselessly paying for it.
So, much better to have NPUs be made separately in different tiers, similar to what is done with GPUs rn.
And even external (PCIe) Graphics Cards can be thin and light instead of being a fat package. It’s usually just the (i) extra I/O ports and (ii) the cooling fins+fans that make them fat.
I want to run LLMs locally, or things like TTS or STT locally so it’s nice but there’s no real support rn
Most people won’t care nor use it
LLMs are best used when it’s a user choice, not a platform obligation
I guess an NPU is better of being a PCIe peripheral then?
And it can then have their specialised RAM too.
Sorry, I’m not a hardware expert at all
When you’re talking about the PCIe peripheral, you’re talking about a separate dedicated graphics card or something else?
I guess the main point of NPUs are that they are tiny and built in
Yes, similar to what a PCIe Graphics Card does.
A PCIe slot is the slot in a desktop motherboard that lets you fit various things like networking (ethernet, Wi-Fi and even RTC specialised stuff) cards, sound cards, graphics cards, SATA/SAS adapters, USB adapters and all other kinds of stuff.
GPUs are also available built-in. Some of them are even tiny.
Go 11-12 years back in time and you’ll see video processing units embedded into the Motherboard, instead of in the CPU package.
Eventually some people will want more powerful NPUs with better suited RAM for neural workloads (GPUs have their own type of RAM too), not care about the NPU in the CPU package and will feel like they are uselessly paying for it. Others will not require an NPU and will feel like they are uselessly paying for it.
So, much better to have NPUs be made separately in different tiers, similar to what is done with GPUs rn.
And even external (PCIe) Graphics Cards can be thin and light instead of being a fat package. It’s usually just the (i) extra I/O ports and (ii) the cooling fins+fans that make them fat.