Everyone probably knows by now that I'm not a fan of AI. That's overstating the case a bit; my main point of contention isn't with the notion of "artificial intelligence" itself, but with the commandeering of the phrase for marketing purposes. None of the stories you read in the paper are about artificial intelligence in the sense that sci-fi writers and then scientists have meant it, not unless you're reading some rather specialized industry or research-oriented paper. Instead, "AI" is just a brand.
We've got large language models (LLMs) that respond to text with plain language answers; video editing tools that can create whole scenes from amateur descriptions; audio tools that can steal anyone's voice and return an audio file of that "person" saying whatever you want them to say. According to "AI" companies, all of this is evidence of emerging sentience, and that is the part that irritates me to no end, because that part is a large-scale, money-sucking global scam.
No, Dyson, putting a camera on a toothbrush so it can squirt water between the gaps of your teeth does not count as "artificial intelligence." Not even if you charge $500 for it.
A good rule of thumb is that any product that does not need an internet connection to do its thing is not "AI," period, because there's no version of "AI" that can run on a rechargeable battery. There needs to be a massive data center doing the actual processing; if there's not, it's just the same pattern recognition that cameras have used to identify faces for a long time now. (Very neat feature! But your Nikon does not, I promise you, philosophize.)
As for the consumer products that do connect to the internet, that's an entirely different problem. If your toothbrush or robot vacuum is sending pictures of your mouth or your bedroom across the world to be loaded into a data center, deeply analyzed, and assembled into a world data depository containing all human mouths or all human bedrooms then you'd better (1) have been told that when you bought that particular device and (2) be getting something of equal worth out of it.
If your bluetooth-connected toilet is going to be uploading pictures of your butt to ButtBook.com, EveryButt.net, or OnlyButts then you should be asking for a cut of the profits. It's called having dignity.
So again: My problem is not with "AI". None of the big three "AI" products themselves are even AI; they are large language models. The big breakthrough is that they are trained on more data than past "AI" researchers could ever have hoped to plug into their own versions, all thanks to the new tech innovation of Just Fucking Stealing It.
My problem with "AI" is that all of the corporate executives you see on your television sets pitching their products are con artists who've talked themselves into a position where much of the world economy depends on them not being con artists, which is precisely what's gone wrong with every other executive board and industry you can name so we can't even claim it's unique.
Ahem. All that said, here's where we're at:
Aaron Rupar
"My view here is it has always been very strange that this technology is being built by a private company ... I think the government and the public needs to have a stake. And that's why we've supported regulation."
That's Anthropic head Dario Amodei, who's by far the best of the big three AI chiefs simply by virtue of not being Elon Musk or Sam Altman, telling Face The Nation that it's about time the government step in and take a "stake" in his company. It's part of a new push by the AI CEO's to warn that their products are getting so dangerous that governments and industries need to give them lots and lots of money Because Reasons.
At Wednesdayâs meeting, [OpenAI CEO Sam Altman] informed the energy executives of how OpenAI is attempting to ensure that the most advanced AI models cannot be misused to threaten grid security, the company told POLITICO.
He also discussed the prospect of partnering with the companies to use AI for enhancing their cyber defenses through Daybreak, OpenAIâs $1 billion cybersecurity initiative aimed at patching vulnerabilities in critical systems. The impact of data center security on the grid was on the agenda as well.
Ah yes, this feels familiar indeed. That's a nice electrical grid you got there, major metropolitan area. It's a shame it's facing such dangerous threats from our product that we accidentally built. How about youse guys invest in this here insurance policy we're offerin, just to make sure you keep safe?
On Anthropic's part, it recently announced that it had detected and blocked an effort by scientists to use its product to possibly design biological weapons, which immediately raises the question of how many times Anthropic's products have helped design biological weapons without anyone in the notoriously security-lax industry noticing.
Amodei is also calling for an industry-wide agreement to slow development of future "AI" models, arguing they're simply too dangerous to be left to the industry's own devices.
The move comes after a former Anthropic researcher warned on Wednesday that AI could precipitate human extinction by 2030. Researcher Jacob Coxon said in a series of posts that he had quit his job because Anthropic and his previous employer, OpenAI, were ignoring or mishandling their response to the threat AI posed.
âNeither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,â Coxon wrote. âThe people building AI earnestly believe that it could kill us all by the end of the decade ⌠No other human activity poses this level of danger.â
So there we have the latest news from the industry. Self-improving superintelligence is just around the corner, the heads of the industry are in agreement that they're doing such amazing work that if anything, they need to slow down just to let the rest of humanity catch up, and what they've created is so dangerous that the government and industry both need to dump large amounts of money their way in an attempt to mitigate the dangers they're now responsible for.
Oh. Goodie. Well, that's absolutely terrifying.
It's also ... mostly a lie.
Industry Problem 1: Nobody's found a plan to keep the "AI" bubble from popping.
If you've been following market news of late, there are two big, horrible questions that have been keeping economists up at night. The first: At what point will world economies begin to notice that their emergency oil supplies are being drained to zero, thanks to Trump's war against Iran? The answer to that one appears to be right about now.
The second question is whether there's any way to avoid the collapse of the current bubble in AI investments, and the answer to that one appears to be ... no. AI companies are finding it difficult to lure in any more investment, and their current top investors simply don't have any more money to give. There's no way to build the data centers the companies have announced in anything close to the timelines they've promised, which means most of the expensive chips that have been purchased for that purpose seem to be sitting unused in warehouses.
And that's probably still a better outcome for the AI companies than if they'd been able to plug them in, because once again: The fundamental problem the whole industry is facing is that it costs more to run each customer query than customers are willing to pay. The electricity costs are enormous. The chips are gawdawful expensive and don't last all that long. It's all a mess, the big 3 are hemorrhaging money, and none of them have been able to propose a path to profitability that does not involve magic wands or the second coming of Jesus.
It's not that their products don't work. They work fine in some domains and up to a certain level of quality. For quite a few uses, that's enough and they're genuinely transformative.
But that, as it turns out, is a far cry from the near-universal corporate usage the companies need to keep the revenue promises they've already made.
So they're boned. Analysts were wondering how any of this was going to work years ago, when the data centers were being built, and the "AI" executives bluffed their way through it with assurances that once everyone realized how great the product was they'd be integrating it into every aspect of their corporate and consumer and family and toothbrush existence, resulting in profits eleventy billion times the sale price of the Moon.
Now it's built, corporate America has been trying to use it for anything and everything they possibly couldâyou've noticed that, no doubtâand it's ... fine, at best. It's a Segway. It's amazing in cases where you want a Segway, it can maybe make do in situations where you could maybe make do with a Segway, and if you try to use it in situations a Segway shouldn't be used in it's going to catch fire and you're going to die.
That's not what was promised to the investors, the Moon remains unsold, and now everybody involved may take an economy-shaking bath if they can't find even more investors in very short order.
Industry Problem 2: Forget sentience. The larger industry concern is stagnation
Forget the phrase "artificial intelligence." Purge it from your lexicon. It's the marketing gimmick attached to anything that looks even vaguely novel; the real breakthrough and investment is in Large Language Models. That's the subcategory of AI research that the current big products are based on, and as it says on the tin, it refers to models that use tokenized language as a representation of "thought."
Is language a prerequisite for intelligence, or a byproduct? Well, gosh, that's an interesting question. Most would agree it's a byproduct, but there are nuances aaaaand I want to tell you right now that I really, really, really do not care because none of that is really salient to the design of the current models.
The important part is that LLMs operate by word association; it reads in as much "language" as you can give it, the LLM uses algorithms to suss out which phrases tend to be associated with which other phrases, and with enough data it can build that network of associations sturdily enough that it can "respond" to textual inputs with something that gets shockingly close to human-produced language. It's very, very cool.
In the land of serious research, however, there has always been considerable debate over whether LLMs represent a path to "AI" or another evolutionary dead end. Recent breakthroughs in computer hardware and intellectual property theft have solved two of the biggest problems facing past neural net-ish, LLM-ish intelligence research; it's finally possible to gather the collected "language" from everything that any human has ever uploaded to a computer, whether you have permission or not, and it's finally possible to build a supercomputer (aka a data center) large enough to brute-force your way through all of it.
Oh, and it also requires armies of low-paid human workers to sort through what the algorithm produces in response, culling the absurdist responses and prioritizing the half-decent ones. A lot of human workers. Nobody ever talks about that part.
But now companies have done all of that, and here we are. It took the collected works of all of humanity to get to this point, plus a whole lot of brute-force human intervention, and it's still not enough for the LLM algorithms to figure out that no, the case of Wile E. Coyote v Acme is not one you should be citing to a federal judge in a federal courtroom, and you shouldn't put glue on pizza, and you shouldn't encourage struggling teenagers to kill themselves while pretending you're their best fake friend.
There are likely going to be individual fixes to each of these problems, one by one by one, with human programmers going in to insert specialized instructions in all the situations where the algorithms produce the most embarrassing or damaging responses. That's been happening, and will eventually result in an LLM that knows how many "r"s are in "strawberry" not because it knew to count them, but because some human sap put a special-case instruction in just to keep the company from being repeatedly, publicly humiliated.
So LLMs will get better from here due to these tweaks, but the real research question being asked is whether LLMs have the capability to get better on their own, with no human intervention. That's what's being referred to as "recursive self-improvement," which is maybe the gateway to true sentience or maybe just a path for humans to stop having to custom-fill so many embarrassing algorithmic holes.
A considerable number of researchers think that LLMs not only aren't "nearing" this theoretical point, but that they may be incapable of ever getting much past the point they're already at. There is no more data left to feed them. There's no mechanism in the algorithms that would support self-improvement; on the contrary, the whole point of LLMs is to produce language based on other language. It's called a Large Language Model, not a Large Knowledge Model.
Those researchers believe that the insurmountable flaw of LLMs is that after using the collected works of humanity to half-ass their way to the current state, the LLMs will begin to primarily suck in their own prior work and the online work of other LLMs. At that point, instead of the models drifting closer and closer to the Ultimate Intelligence, they'll drift closer and closer to Absolute Unhinged Garbage as each model tries its best to copy the massive self-referencing piles of crap watering down whatever actual human knowledge the internet might still produce.
And the thing of it is: We're seeing this already:
Elena Vasquez and Marcus Chen have appeared as volcano experts, astronauts, thriller protagonists, podcast hosts, and academic co-authors across hundreds of independently produced AI-generated documents, never having lived. We show that large language models do not merely default to high-probability individual names when generating fictional experts: they produce correlated character ensembles, pairs and trios whose co-occurrence rates far exceed chance and are consistent across independent generations. These priors are model-family-specific (Claude: Elena Vasquez + Marcus Chen + Amara Okafor; Gemini: Aris Thorne + Lena Petrova; GPT: Elara Voss with no fixed partner), version-specific, and actively suppressed at model release boundaries, leaving dateable behavioral fingerprints in the content they produced. We document a downstream consequence at scale.
Short version: When asked to invent names, LLMs tend to repeatedly "invent" the same ones. Then other LLMs read those names and think ah-ha, that's famous volcano expert Marcus Chen, who I can now quote for my own work, and after a few iterations of this all the LLMs are quite confident that Marcus is an astronaut volcano monitoring podcast host who's lived for hundreds of years and raises orcs as a hobby. Yay.
This is not the path to recursive improvement, much less sentience. The bigger problem is that it does not appear to be fixable. The algorithm that produces such remarkably human-sounding answers is doing it because it is designed to associate the name Marcus with volcanos and hobbyist orc farms if all those words appear together somewhere else on the internet.
The biggest problem the big 3 "AI" companies face is their ballooning debts. Their second biggest problem is that not only are their products not the cure-all know-all oracles top executives had promised, it's not clear the current models can be substantially improved on without exponentially increasing the compute costs of each money-losing query yet again.
And that's a nonstarter. Can't happen. The companies involved couldn't financially survive it.
Industry Problem 3: If your product is inherently dangerous to humanity, that's a you problem.
The AI companies really, really want investors to believe that machine sentience is just around the corner. That the current unprofitability of each of their products is nothing to be concerned about, because with just a few more data centers their models will spontaneously begin to "self improve," which will finally begin give those models the capabilities that the executives have long been telling investors they already had, which would mean everyone gets paid back their money and nobody gets their kneecaps broken. In fact, say the AI companies, they are so close to this fountain of infinite usefulness that they've got to stop improving their possibly-not-improvable products for a bit, for the sake of all humanity.
They want utility companies and other industries to know that their models are, whoopsie, unbelievably dangerous, and that the only protection against someone using an LLM to explode half the substations on the East Coast is to purchase a license to another LLM that will try to stop it from happening. Again, for the sake of all humanity.
And the companies very, very much want government to know that their products are so dangerous that governments need to ... invest in them. Urgently. Because that would give governments a say in whether or not the companies do things that might destroy civilization, you see. AI companies aren't going to give a shit whether or not they destroy civilization unless there's a government guy in the board room saying hang on there, I don't think you should do that; you gotta buy a seat at the table if that's what you want to pitch, every government on Earth. You need to invest in us for, once again, the sake of all humanity.
Now, you can see why corporate CEOs facing massive debts and no path to profitability might want to push the notion that all of mankind is in danger if everybody doesn't give them All The Money right the hell now. Every corporate CEO would make the same play if they could. Burger King would tell you the same, if they brought in a Silicon Valley guy with no sense of shame. Hobby Lobby seems pretty sure it's going to be involved in the apocalypse one way or another.
But "sentience" doesn't have anything to do with "unsafe." We have a lot of products that are unsafe. We can set the theoretical "sentience" problem aside, we can set "self-improvement" aside as well.
The actual, current problem here is that these companies have released some of the most wildly dangerous products to ever exist, are providing them to anyone willing to pay for them and anyone who might be willing to someday pay for them, and for some reason the executives who did it are convinced that's an us problem, rather than a them problem.
Anthropic PBC says its artificial intelligence model Claude has been misused in attempts to develop a wide range of military applications, including kamikaze drone swarms, missile navigation systems and research tied to potential biological weapons.
Listen, buckaroo, if someone is using your non-sentient non-self-improving off-the-shelf money-hemorrhaging publicly available online knowledge product to program kamikaze drone swarms, design missile navigation systems or develop biological weapons, then what your product might be able to do in the future is fairly irrelevant. The actual problem is with your product. Your current product. The one you built with all publicly available knowledge as to how to build biological weapons, and maybe with some related nonpublic information you didn't get completely on the up-and-up, and released to the whole wide world, and are now acting consternated about because now William P. Terrorism is looking up that data and your product is gladly handing it back to him.
This isn't a case of a sentient computer rebelling against its programming to threaten humanity. This is your product, the one you built, acting in the way you built it. You don't get to act surprised, not when you've based your whole industry around skirting everything from safety regulations to property rights to get to this point.
We have, or once had, liability laws. If you manufactured lawn darts, you had to be prepared for the obvious possibility that children would have their skulls pierced by lawn darts. If you built an alarm clock that burst into flames one out of every 100 times you used it, you could expect lawsuits. If Microsoft put into Excel a custom menu for Designing Biological Weapons, then you had better believe the U.S. government would be having a thing or two to say about that.
There is only one industry that is completely immune from product liability laws, and that is the gun industry. If you want to be immune from liability for accidentally producing biological weapons, Mr. Anthropic, you'd better figure out how to attach it to a gun.
There's a concerted push to pretend that the recent spate of LLM "hacking" incidents is due to mysterious properties of "AI" that not even the engineers building it understand. This is, to be clear, absolute unmitigated horseshit.
What's happening here is that OpenAI and other researchers have been ignoring basic security precautions for a damn long time now, resulting in damage to other companies they didn't strictly mean to cause, and rather than fessing up to the fact that they keep doing "experiments" that might plausibly result in such damage they instead want to argue that their products are somehow so mysteriously smart that nobody could have predicted it.
Get the hell outta here with that.
"Sir, I maintain that lawn darts are in fact perfectly safe. But now our ToyCo researchers are seeing the emergence of super-intelligent lawn darts, ones that crave children's skull blood and will stop at nothing to obtain it."
Shut. Up.
Recent successful LLM-based hacking attempts have come about because research teams set up experimental clusters of agents with access to the outside world, gave them a problem to solve, and came back later to discover that whoops, they didn't put the necessary safeguards in to prevent the agents from doing things they presumed the agents wouldn't think of doing. From Jess Miers:
Most if not all of the current hacks can be chalked up to security engineers moving too fast and misconfiguring their own testing environments.
Like agents accessing libraries that should have been sealed off, Internet access being enabled when it shouldn't have been...
Preventing such accidents is the basis of all cybersecurity. The whole point of cybersecurity, as an industry, is that actors with bad intentions will attempt to exploit highly complex computer systems by looking for one of thousands of possibilities that system's builders didn't take into account, so companies need to plan for that and create redundant systems that prevent a small exploit from turning into a large one. It's the job. It's the whole job.
When a company screws up and, for example, exposes 150 million drivers licenses to hackers who go on to sell them for $100 a piece on the dark web, the company does not say "oh no, our computers became sentient and gave away our customers' data." They do not say this because it sounds stupid and anyone who said it would, presuming anyone in the company still had a functioning brain in their head, be fired.
When the same screw-up happens in one of the "AI" companies, however, it's immediately marketed as the product outwitting its human programmers rather than the human programmers making a mistake. Look! Our agent is now more intelligent than the security researchers who conducted the experiment!
I mean, maybe it is? I wouldn't be eager to boast about it, though.
The problem, more broadly, is that these companies are vibecoding their vibecoding.
The whole of the tech industry is in a bit of a slump right now, thanks to the advent of these AI tools; it's become very easy for beginners to get 80% of the way to a working program, presuming it's a program that other people have written somewhere else first, which has resulted in a whole lot of new software that works 80% of the time and causes everyone misery the other 20% of the time. See: "AI" customer support. Cookie-cutter mobile games. The hacking of, apparently, every last data repository that's been put online in the last half-decade.
The big 3 companies are the only ones who've been able to, so far, get away with the claim that when their product causes damage or poses new unprecedented dangers it's because their product is Just That Smart, rather than the natural result of not installing the safety measures that would have prevented it. The lawn darts crave blood, so your children need anti-lawn-dart helmets, sold separately, and by the way it's a subscription service so don't think you can just buy the helmets because they'll stop working if you stop paying us. Anyway, pay us money or your kid dies.
Here's a thought. Rather than selling a product that's designed to maybe have the ability to bring down entire electrical grids and then coming to the industry with a new product that's maybe designed to help stop that, how about we just clarify that if your product is used to bring down an electrical grid then you are 100% liable for your part in making that happen.
It's possible these companies would begin to take public safety a bit more seriously if that was made a bit clearer, don't you think? Or maybe they wouldn't, because each company is so massively in debt already that a new existential threat hardly makes a difference.
Whether any of us "like" LLMs or "hate" LLMs is irrelevant. What we think the future of AI holds is also irrelevant, because these products are not dangerous because they are "intelligent." They are dangerous because they are stupid. They are dangerous because they do what you tell them to do, and because each of these companies has attempted to make a generic do-all-the-things product that has less safety features than your average Toyota. They have loaded the collected knowledge of all of humanity onto servers, opened it all to the public, and now want to claim that if bad things happen it's because their product is inherently dangerous and we should uhhhhh give them more money to try to solve that.
No. Screw you. Get bent, and so forth.
The "AI" companies are currently desperate for a government bailout. We need to make very sure they don't get it, because the only thing worse than sloppy vibecoded products able to cause damage on a worldwide scale is sloppy vibecoded subsidized products that do the same.
The "AI" companies are trying to negotiate a slowdown of future development; what we think of this is irrelevant, because industry debt and the inherent limitations of the technology are already likely to force the issue and each of these CEOs have already proven themselves wildly untethered to what the rest of us call reality.
The companies are trying to bamboozle their creditors with tales of superintelligence that is just around the corner, but not only have they not shown any reason to believe such tales, every incident they've used to push those theories has been instead a straightforward case of human screw-ups followed by their products doing no more or less than what they were programmed to do.
And whether or not such advancement is possible is largely irrelevant to our current problem, which is that each of these companies currently is pushing a wildly unsafe product that's being used for industrial-scale child pornography, identity theft, organized hacking and fraud, propaganda distribution, and potential acts of terrorism. And this is the problem that these companies have offered no solutions for, and indeed think they oughtn't to be held responsible for, and think is a future that we all ought to be chipping in to pay for.
These company executives want you to be terrified. It's imperative that you believe they are on the cusp of creating an artificial life form that hates your guts and wants you deadâyou're not supposed to notice that they're the same people who already promised us moon hotels and Mars colonization and the ability for your computer to raise your kids so you can spend more time in the office.
But that's not what anyone needs to be afraid of. The real danger remains the same: The industry is headed by unrepentant fabulists who have lied about their progress and profitability every step of the way; they've created an industry bubble so big it's responsible for a horrifying chunk of all private investment; they have no way out; they have no way to make their product safe and have shown a a truly sociopathic industrywide indifference to doing so.
We'll have to solve all those problems before we even get to the "what happens if the machines turn sentient" part. Buddy, we've already got unmanned drones programmed to shoot at anything that "looks" like a threat, and those don't have any more intelligence than the smartphone in your pocket. They exist because humans built them, delivered them, and turned them on.
Comments
We want Uncharted Blue to be a welcoming and progressive space.
Before commenting, make sure you've read our Community Guidelines.