I enjoyed Sabine’s analysis in another video that continuing to make increasingly larger models with more compute is about as effective as continuing to make larger and larger particle accelerators. Come on, bro, this million km Gigantic Hadron Collider will finally get us to the TOE. Just one more trillion, bro.
Every step towards the next generation of colliders needs to be deeply justified about the falsifiables it will check and their interest to the current knowledge before being able to see a cent for it, and the expected energies of the TOE are well known to not be reachable with current means and technology, that’s not what they are promising ever, but what they do they fulfill, often, beyond predictions, to not mention the huge return basic research has always had in the long term to humanity… nope, I am afraid that I do not find it a good analogy at all.
EDIT: but, yes, such strategy of making it bigger does not work anymore, so collider proposals go usually in other directions…
Every particle accelerator that has been built has paid for itself in research value. There’s basically nothing that comes out of AI research except the need for a bigger model.
The comparison is poor. Particle accelerators are science, LLMs do not produce science.
That’s not to say that we couldn’t build LLMS that would be useful for scientific purposes but we’re not. That is not the function or the goal of the people building these things.
TL;DR - Many times the cost of the LHC and unlike the LHC, the gains are likely to be incremental instead of revolutionary. The same funding could do much more good elsewhere.
To your point, agreed that even small, incremental gains for science are more valuable than what we are likely to get from AI.
Except that’s not at all what they’re doing. Most of the impact studies are already outdated, and the models are shrinking and becoming more efficient.
Used to love Sabine, but the channel’s been taken over by sloppy clickbait.
I enjoyed Sabine’s analysis in another video that continuing to make increasingly larger models with more compute is about as effective as continuing to make larger and larger particle accelerators. Come on, bro, this million km Gigantic Hadron Collider will finally get us to the TOE. Just one more trillion, bro.
Every step towards the next generation of colliders needs to be deeply justified about the falsifiables it will check and their interest to the current knowledge before being able to see a cent for it, and the expected energies of the TOE are well known to not be reachable with current means and technology, that’s not what they are promising ever, but what they do they fulfill, often, beyond predictions, to not mention the huge return basic research has always had in the long term to humanity… nope, I am afraid that I do not find it a good analogy at all. EDIT: but, yes, such strategy of making it bigger does not work anymore, so collider proposals go usually in other directions…
Every particle accelerator that has been built has paid for itself in research value. There’s basically nothing that comes out of AI research except the need for a bigger model.
The comparison is poor. Particle accelerators are science, LLMs do not produce science.
That’s not to say that we couldn’t build LLMS that would be useful for scientific purposes but we’re not. That is not the function or the goal of the people building these things.
Not really my area of expertise, but this article lays out her perspective on this for anyone who isn’t aware: https://www.scientificamerican.com/article/the-world-doesnt-need-a-new-gigantic-particle-collider/
TL;DR - Many times the cost of the LHC and unlike the LHC, the gains are likely to be incremental instead of revolutionary. The same funding could do much more good elsewhere.
To your point, agreed that even small, incremental gains for science are more valuable than what we are likely to get from AI.
Except that’s not at all what they’re doing. Most of the impact studies are already outdated, and the models are shrinking and becoming more efficient.
Used to love Sabine, but the channel’s been taken over by sloppy clickbait.