Sam Altman:
A lot of people want local inference, and somehow they assume that the problem is solved now because we have open-source models, we’re going to be able to run them on our computers, and so we’re not going to need all the stuff in the data center.
But from a global allocation perspective, actually it’s quite terrible to have everybody, like, have a bunch of the world’s limited supply of DRAM sitting in their own computer and being used five percent of the time.
Like you want maximum utilisation in the cloud if you’re trying to deal with this very supply-constrained environment,
And somehow many people’s intuition is the opposite of that
TL;DR Guy selling product: “Here’s why owning your own product instead of renting it from me is a terrible idea”
A lot of people want to be billionaires, and somehow they assume the problem is solved because we have unfettered predatory late stage capitalism and oligarchy in America, we’re going to be able to amass as much money as we want in our bank accounts, and so we’re all going to be rich without oversight, accountability or government interference.
But from a global allocation perspective, actually it’s quite terrible to have everybody, like, have a bunch of the world’s wealth sitting in a few private bank accounts and being used .05% of the time.
Like you want maximum circulation of currency if you’re trying to reduce inequality in this artificial scarcity environment,
And somehow, Americans’ intuitions are the opposite of that.
FTFY Sam.
This is genuinely a fantastic response that is also absolutely correct in its premise
Funny because it’s true
Fighting so hard not to make an edit about raping sisters, but that might be too degenerate for me
Ram is so cheap to make that until the AI demand companies basically had to create new levels of density/speed and sell for less just to keep the business afloat. This is like when cellphone companies tried to claim megabytes were a precious finite resource. We live in a post scarcity society and these billion assholes are like “well that just won’t do”.
It reminds me of that kid we all know who just learned economics from school house rock and then goes “now if I can simply find every evee Pokémon card in the world and burn them all except for the one I have, I’ll be rich!”
It’s just so frustrating to see this sheer stupid amount of greed all of the time. Thanks for coming to my TEDex talk.
Even if it was a scarce resource that’s important to use efficiently, p2p inference sharing/markets are a thing, local hardware ownership doesn’t have to mean it all sits idle.
Next he’s gonna say people should stop using water.
like when cellphone companies tried to claim megabytes were a precious finite resource
Limiting the usage of a shared channel is a valid approach, it’s remarkable how so many people just absolutely refuse to understand this. You can use the entire bandwidth of the shared channel for a while, when your webpage is loading. If you use the entire bandwidth nonstop, you fuck over everyone else on the channel. So the company has the choice between giving you fast load times but limiting your usage otherwise, or giving you 32 kbps unfettered.
If you want an entire channel of guaranteed bandwidth with unlimited usage, you always could and can get it — under business plans, which are orders of magnitude costlier, because they don’t share the channels.
In a similar fashion, if your healthcare system is entirely free, it’s getting clogged with retired old people who are lonely and have nothing better to do. If a doctor visit costs even a couple bucks (depending on the region), the doctors are suddenly much more free to deal with real cases.
Right… In this instance I would argue openAI is the shared channel. There are three memory manufacturers now illegally rigging the market as a push back against the corporate AI. Something like a memory oligopoly VS ai monopsony?
If we all have inference, we avoid clogging shared channel.
And as for megabytes, the value of the megabyte isn’t a finite resource and the value of it is determined by what data is served in the megabyte and the cell companies didn’t care what megabytes you’re getting charged for (example full bitrate advertisements on YouTube). I still think that’s a valid argument to be had.
By the value based on scarcity of bandwidth logic, the phone companies would be able to charge peak megabyte hours to specific websites (again like they used to for telephone calls with plans that included free calls at night) when in fact they can handle much more bandwidth now. In reality when it comes to that traffic, it’s often the providers at the ends of the line who have to suffer the constraints of demand. Maybe they should charge per minute of video accessed or it could be a solved problem in a hyper-competitive, over-saturated marketplace.
I understand what you are saying. I just don’t believe it applies in this specific domain any longer. This is potentially a more valid take an underserved market.
the phone companies would be able to charge peak megabyte hours to specific websites
ISPs in the US were aiming to do that, since video traffic dominates the web pretty heavily now. But that could be easily abused, so they were slapped away with the net neutrality regulations, which iirc Trump rolled back.
I just don’t believe it applies in this specific domain any longer.
You referred to the megabytes thing in the past tense, yes, but the reasoning applied back then. In fact, gigabit speeds are still not prevalent worldwide (nor just in the US, from what I understand about their shitty ISPs). I can easily saturate the 100 mbps allotted to my cable-connection plan in the neighbourhood that doesn’t provide much choice, and I’m pretty sure mobile connection isn’t better than that in most cases.
No… he actually is right. A local machine will sit idle most if the time.
A data center can balance loads globally.
In terms of resources… data centers ARE cheaper.
However … there are other reasons to use local interference that might be more important than added cost
Totally agree. It’s really about wealth distribution and who controls the resources. Altman is just trying to steer the conversation toward efficiency instead.
Well yes, or data security or whatever.
But on a purely costs level such big data centers are cheaper… especially since they certainly getting better discounts than anyone else.
No, his analogy is poor for many reasons. One is that people using local models are likely using that computer for other things as well, it is a resource that does many things unlike their slop farms. Also, DRAM wasn’t a “scarce resource” until they devoured the industry with their AI ponzi scheme.
Only very few people have enough processing power to run a proper model.
But even then I’d rather rent capacity from a hyperscaler than pay him.
Going cheap almost always ends in a more expensive repair bill down the line.
Likewise, doing the most convenient thing to satisfy one’s own desires is probably the most ill-advised course of action. Mainly, because it makes other people take the most convenient course of action to remove the other person from being a problem in their lives altogether.
Sure… locking your customer in is always a good business option.
But the point remains… for any given demand a big data center always is cheaper.
Cheapest doesn’t mean best for all uses cases.
That’s insanely ironic coming from him. Meanwhile the crazy amount of waste that these companies are producing buying all the GPUs to create artificial scarcity. They are terrified of people being able to run their own local LLMs because it completely destroyed their entire business model, they will never be profitable.
new title:
“Man whose company lost $20b in previous year explains how to allocate resources.”
A lot of people want local inference, and somehow they assume that the problem is solved now because we have open-source models, we’re going to be able to run them on our computers, and so we’re not going to need all the stuff in the data center.
We don’t need them even without local inference. There was already more than enough total global compute for the most valuable use cases of LLMs (research, development, admin). The datacenter build-up is almost entirely fueled by utterly unrealistic expectations of future capabilities. Before and now the case for local inference is a matter of access, not capacity.
But from a global allocation perspective, actually it’s quite terrible to have everybody, like, have a bunch of the world’s limited supply of DRAM sitting in their own computer and being used five percent of the time.
Utilitarianism like this is a disease. By the same logic, it’s inefficient that we allow other species of animals to exist and consume usable energy. Hell, why stop at other species? It’s inefficient to allow humans to exist - let’s kill all humans!
Now, I know that’s not what he said, the point was to show that when someone uses utilitarianism as a serious argument, they are no longer worth listening to.
Like you want maximum utilisation in the cloud if you’re trying to deal with this very supply-constrained environment,
The supply is only constrained because you keep hoarding more than you need. Maybe it’s time for some communization? So communities where data centers are built get most if not all of the profits from it?
It’s inefficient to allow humans to exist - let’s kill all humans!
That’s already where his mind is at: https://www.theregister.com/software/2026/02/23/altman-you-think-ai-is-wasted-energy-try-raising-a-human/4181781
Well the less humans that are kept happy by having enough ram to do stuff the more unhappy humans looking for someone to blame there are… But yeah what’s the point in anything if there’s nobody there?
At this point - especially with this commentary - the datacenter build-up seems like market manipulation. Like how hyper-scaling keeps competition scarce.
I bet he owns a car that gets less than 5% usage.
Having billionaires is a terrible allocation of scarce resources, don’t you find?
I came here to say exactly this!
Change the word DRAM to car and see how ridiculous this sounds. And I’m a big supporter of public transit.
distributing trust to every person on the planet is so inefficient. All trust should be given to me!
CEO of Pizza Hut says making pizzas at home is a terrible allocation of resources.
Computing for me, not for thee
Local models would erase 2 trillion dollars from the market so I would think that Sam Altman isn’t a fan of the concept.
letting sam altman (or billionaires for that matter) have all the money from our work is a poor allocation of resources.
Guy selling product: “Here’s why owning your own product instead of renting it from me is a terrible idea”
Guy with the spying infrastructure in place to steal your product when renting his product, says you escaping his ability to spy is waste of resources.
But Sam, local inference is what would prevent our data from being used by creepy advertisers.
we lost Captain Picard yesterday, yet this lazy-eyed cardboard human gets to keep using oxygen. there is no justice in this world.
Fucking idiot is too stupid to realize that local DRAM can be used for useful shit, as opposed to whatever he wants to use it for.
