What on Earth are all these companies doing with racks and racks of hardware? Are they somehow constantly finetuning models for their workflows… for what?
The Chinese are pretraining, finetuning, and publishing with (comparative) peanuts for hardware. Some random company’s local pretrains are not going to beat the leading edge of open source.
And finetuning should usually be a one-off cloud runs, not something companies horde hardware for.
I thought it was just my company who made some major investments in server hardware.
They put it under the guise of LLMs. But I’m also wondering if it’s because of the US government’s overreach in the past year. CEOs who own Cloud infrastructure have continued to kiss Trump’s ass, and on-prem is now the only thing we can trust.
LLM inference requires very specific servers that aren’t good for much else (in terms of what companies usually do), though. And go ‘obsolete’ even more quickly.
I guess what I’m saying is the premise would be pretty flimsy for a more general upgrade.
Yeah.
What on Earth are all these companies doing with racks and racks of hardware? Are they somehow constantly finetuning models for their workflows… for what?
The Chinese are pretraining, finetuning, and publishing with (comparative) peanuts for hardware. Some random company’s local pretrains are not going to beat the leading edge of open source.
And finetuning should usually be a one-off cloud runs, not something companies horde hardware for.
I thought it was just my company who made some major investments in server hardware.
They put it under the guise of LLMs. But I’m also wondering if it’s because of the US government’s overreach in the past year. CEOs who own Cloud infrastructure have continued to kiss Trump’s ass, and on-prem is now the only thing we can trust.
LLM inference requires very specific servers that aren’t good for much else (in terms of what companies usually do), though. And go ‘obsolete’ even more quickly.
I guess what I’m saying is the premise would be pretty flimsy for a more general upgrade.