This specific GPU is… Kind of a mixed bag. It’s supposed to be built on a 6nm process, and the G100 is, according to Lisuan, the first domestic chip to genuinely rival the NVIDIA RTX 4060 in raw performance, delivering 24 TFLOPS of FP32 compute. It even introduced support for Windows on ARM, a feature even major Western competitors had not fully prioritized.
It appears to fall short of its marketing promises, though. An alleged Geekbench OpenCL listing revealed the G100 achieving a score of only 15,524, a performance tier that effectively ties it with the GeForce GTX 660 Ti, a card released in 2012. This places the “next-gen” Chinese GPU on par with 13-year-old hardware, making it one of the lowest-scoring entries in the modern database. The leaked specifications further muddied the waters, showing the device operating with only 32 Compute Units, a bafflingly low 300 MHz clock speed, and a virtually unusable 256 MB of video memory. We’ll likely see more benchmarks as the GPU makes its way to the hands of customers.
These “anemic” figures might represent an engineering sample failing to report correctly due to immature drivers—a theory supported by the test bed’s configuration of a Ryzen 7 8700G on Windows 10. But still, if true, the underlying silicon may still be fundamentally incapable of reaching the promised RTX 4060 performance targets, regardless of the actual specifications that are being reported.
Which seems wildly shortsighted, like surely the AI space is going to find some kind of more specialized hardware soon, sort of like how crypto moved to ASICs. But I guess bubbles are shortsighted…
The crazy part is outside LLMs the other (actually useful) AI does not need that much processing power, more than you or I use sure but nothing that would have justified gigantic data centers. The current hardware situation is like if the automobile first got invented and a group of companies decided to invest in huge mortal engines style mega-vehicles.
Debatable. The basics of an llm might not need much, but the actual models do need it to be anywhere near decent or usefull. Im talking minutes for a simple reply.Source: ran few <=5b models on my system with ollama yesterday and gave it access to a mcp server to do stuff with
Derp, misread. sorry!
Yes, my whole post was that non-LLMs take far less processing power.
Oh derp, misread sorry! Now im curious though, what ai alternatives are there that are decent in processing/using a neural network?
So the two biggest examples I am currently aware of are googles AI for unfolding proteins and a startup using one to optimize rocket engine geometry but AI models in general can be highly efficient when focussed on niche tasks. As far as I understand it they’re still very similar in underlying function to LLMs but the approach is far less scattershot which makes them exponentially more efficient.
A good way to think of it is even the earliest versions of chat GPT or the simplest local models are all equally good at actually talking but language has a ton of secondary requirements like understanding context and remembering things and the fact that not every gramatically valid bannana is always a useful one. So an LLM has to actually be a TON of things at once while an AI designed for a specific technical task only has to be good at that one thing.
Extension: The problem is our models are not good at talking to eachother because they don’t ‘think’ they just optimize an output using an intput and a set of rules, so they don’t have any common rules or internal framework. So we can’t say take an efficient rocket engine making AI and plug it into an efficient basic chatbot and have that chatbot be able to talk knowledgably about rockets, instead we have to try and make the chatbot memorise a ton about rockets (and everything else) which it was never initially designed to do which leads to immense bloat.
This is why i played around with mcp over the holidays. The fact its a standard to allow an ai to talk to an api is kinda cool. And nothing is stopping you from making the api do some ai call in itself.
Personally, i find the tech behind ai’s, and even llm’s, super interesting but companies are just fucking it up and pushing it way ti fucking hard and in ways its not meant to be -_-
Thanks for the info and ill have to look into those non-llm ai’s :)