

If you can make a useful MoE thing where each expert model has a small final layer in its neural net, so you don’t need to move much data between cards, then running each MoE on a different card might be viable. Regardless of whether the GPU vendor wants to segment up the gaming and AI markets.
I think that that’s one of the biggest unknowns as to where AI may wind up going. If you can get good results on gaming cards, then suddenly ordinary gaming hardware, run in parallel, may be quite capable of running the important models, and it’s going to be much harder for OpenAI or similar to obtain much of a barrier to entry. That may have dramatic impact on who has what degree of access to AI.












Ehh. Not sure I agree. I mean, I think that there is a valid insight that it’s important to keep track of what problem you’re actually trying to solve, and that that problem needs to translate to some real world, meaningful thing for a human.
But I also think that there are projects that are large enough that it’s entirely reasonable to be a perfectly good engineer who isn’t dealing with users much at all, where you’re getting requirements that are solid that have been done by up someone else. If you’re trying to, say, improve the speed at which Zip data decompression happens, you probably don’t need to spend a lot of time going back to the original user problems. Maybe someone needs to do so, but that doesn’t need to be the focus of every engineer.
I think I’d go with a more specific “It’s generally better to iterate”. Get something working, keep it working, and make incremental improvements.
There are exceptions out there, but I think that they are rare.
This is one thing that I think that Microsoft has erred on in a number of cases. Like, a lot of the value in Windows to a user is a consistent workflow where they can use their existing expertise. People don’t generally want their workflow changed. Even if you can slightly improve a workflow, the re-learning cost is high. And people want to change their workflow on their own schedule, not to have things change underfoot. People don’t like being forced to change their workflow.
I don’t know if it’s the fastest, but I do think that you often really discover how embarrassingly large the gaps in your own understanding are when you teach it.
A little kid asking “why” can be a humbling experience.