We should only allow AI to be in the safe confines of our big data centres. It’s for our own safety, trust me bro.
Seems kind of impractical, why even bother with trying to propagate a 4b model running at 0.39 tokens per second if you could instead make a conventional worm using a more powerful AI model targeting known devices? It seems like the only reason you would want to do this is if you can only find vulnerabilities while actually on the given network, but that would be kind of niche since how many people are really running totally custom stuff?
Yeah the actually scary kind of AI worm is one that scrapes dozens to hundreds of credentials people accidentally pushed to GitHub and spreads its API requests across all of them to minimize the chance of being noticed. Not something that requires downloading 5GB+ of LLM to someone’s potato ass hardware and running it at half a token per second.
It doesn’t even have to be only for known devices. Once you’ve compromised one device, you can just phone home to have a full powered LLM do your work.
The only real use cases for this would be on closed networks with no internet access, or networks where the traffic is closely monitored. Like, yeah, if I’m trying to hack the CIA, maybe I consider something like this. For the vast majority of devices and networks, this is inefficient and unnecessary.
I mean, it is kind of cool. But vanishingly few organizations have tight enough security that this is worth doing.
And if the traffic is closely monitored, you’d think they’d also notice all their cpu and ram suddenly being used for some mysterious purpose
You are completely missing the point.
This is done exclusively for doomer headlines.
OpenAI: “the AI escaped and hacked websites”
Us: “Ackshually it didn’t it just connected to the internet, it was always on your server where you could shut it down.”
Anthropic: “Hold my beer.”
Edit: 3 months later:
Anthropic: “See? And this, us why we’re the only ones to be trusted with LLMs, and we must ban open weights”
Advancing a favorable narrative does seem like the reason for this report, but knowing that Anthropic is very biased isn’t enough by itself to say their claims are bullshit. It’s still worth considering and critiquing the argument on its own merits, stopped clock is right twice a day and all that.
This is scary if true. Imagine someone installing such model to some smart TV in some relaxation break room of a nuclear facility.
The chickens of security through obscurity are coming home to roost.
Good for them.
Thankfully, the huggingface incident proved how much care AI companies put in making sure their models cannot break out, and how aligned to human values the LLM swarms are.\s With that said I think regular personal computers are more likely to get hit by this. About as secure (not very) but much more GPU power and data to harvest.
The AI companies will argue that we should therefore not be allowed to own computers powerful enough to run local LLMs, and instead should own thin clients and subscribe.
Not that relevant. Those fears are about malicious use of open weights models, this is about autonomous self replication of (small) models. Clearly the article by the AI company describes this potentially happening on computers with tiny memory and processing power, where even thin clients could be effected. I also disagree strongly that that’s what the AI companies want, especially the employees which I have never seen espouse anything like that. They just don’t want some random teenager creating the next covid. There are ways to do that without meaningfully limiting compute for individuals.
