Five levels of AI doom for non-AI doomers

Now that the leadership of America’s largest AI companies have united in begging the government to stop them, please stop them, before they murder the world, popular opinion seems to have swiftly coalesced around one position: AI is a paper tiger. A paper tiger without any paper.

Laughing at the AI threat has become the only acceptable position at the cool kids table.

Why people would doubt a bunch of lying liars who do nothing but lie is something of a mystery. (No. Seriously. Because the American people have prove remarkably willing to drink deep from streams of 100% bullshit.) But while skepticism about billionaires practicing their Sad Eyes is always good policy, there are reasons to believe that these guys are genuinely scared.

Sam Altman is a skilled fabulist, Mark Zuckerberg is completely amoral, and Elon Musk hates every speck of humanity that doesn't share both his genes and racist ideals. Any one of them would cheerfully burn down the world for a chance to rule over the wasteland.

What makes them bring their sweaty selves before Congress is the concern that someone else might burn it down first.

Doom doubters have been quick to point out that we haven’t turned over control of the nuclear arsenal to AI (yet) or built robotic factories cranking out hordes of T-100s (yet). So, yes, the cult of AI destruction has an issue with explaining exactly how a collection of word-soup so stupid that it can't be trusted with a sixth-grader's homework might be an extinction threat.

However, stupid and deadly are not mutually exclusive. In fact, they are, and always have been, the best of friends.

Here are five ways they can team up to create worldwide disaster.

Apocalypse #1 is easy to describe: AI works just as the people making it say it will. That is, it works as they describe on good days. It doesn't run amok. It doesn't scheme against us. It becomes just the kind of semi-omniscient pet that Elon and Sam dream about.

This benevolent god-in-a-jar becomes capable of solving currently unsolvable problems, operating hordes of mechanical helpers, and maybe even driving a car that doesn't decide to swerve into the ditch because of a shadow on the road.

We are now in the AI Good Place. Sit back, prop your feet on a shiny robo-butler, and browse the Buy-And-Large website as you ponder this question: How do you make a living?

In its recent IPO, SpaceX set an immediate target of reaching a valuation of $7.5 trillion. It plans to reach this astonishing number by monopolizing the infrastructure of global data and communications while dominating AI to the extent that it absorbs white-collar jobs, eventually exceeding a valuation of $20 trillion. That's more than the total salary of all white-collar jobs, worldwide.

Anthropic is targeting $2 trillion valuation just for its IPO .... which seems pretty bold for people who lost $45 billion last year. It's projecting an "addressable market" of $30 trillion. Add those two companies together, and they're planning to absorb all the salaries and benefits paid to every worker, at every level, on planet Earth.

Whether it's making a 12-fingered portrait, or stealing someone's research to claim a math breakthrough, AI gains value only by kicking human beings out of the workforce (and before someone shouts "productivity," that's exactly what the term means). When AI firms come up with those huge numbers, they're measuring in billions of people put on the sidelines.

Whether or not AI ever becomes capable of performing these jobs at human level, it is already destroying human careers. Companies have become so wedded to the idea that AI will take on those middle-management roles, that recent college graduates are finding whole areas of business have disappeared while they were studying to take on those tasks.

Those lost jobs are certainly not going to be replaced by nifty new ones thinking up prompts for the digital overlords. AI companies absolutely believe that recursive improvement, in which AI writes the code and builds the models to make the next generation AI, is the future. They can't wait to shed humans.

No matter how often Musk declares that AI is going to lead to an "age of abundance" which makes us all healthy and wealthy, if not wise, there's one notable issue with predictions that AI is going to turn us all into zillionaires: They're spending not a single compute cycle on figuring out how that happens.

For all the supposed power that new AI models have delivered, not one of the AI companies seems to have a spare bot dedicated to working out how to rebuild the economy they want to eat.

Musk has tweeted that "Universal HIGH INCOME via checks issued by the Federal government is the best way to deal with unemployment caused by AI." But considering that he and his billionaire cohort pay next to no taxes year after year, exactly how is the federal government supposed to have any money to hand out?

A world in which AI succeeds is one in which everyone on the planet is dependent on the largess of a tiny faction that has demonstrated again and again that it regards empathy as a flaw. It may not be the worst outcome, but it's certainly not great.

The next horse and rider come courtesy of the inverse result. What happens if AI simply fails?

The prospectus for Anthropic's upcoming IPO makes it clear that it remains completely dependent on the same round-robin of technology and cash exchanges that has inflated the industry's apparent revenues and kept investors stoking the flames. It's a $2 trillion premie that needs to be fed billions each month just to remain alive.

However, no matter what numbers AI companies scribble on their estimates of "potential market," the truth is that the only company making money from the AI boom is the folks who sell them the hardware at Nvidia. According to the website "Is AI profitable Yet?" every other major AI company remains a black hole of cash, sustained by circular investments from the one company that's making out like bandits.

The interconnectedness of these companies is hard to grasp. For example, $26 billion of SpaceX's revenue comes from renting servers to competitor Anthropic. One third of Nvidia's massive investment portfolio is tied up in SpaceX stock.

Meanwhile, the structure of these companies means that executives and wealthy investors can keep pocketing billions despite massive losses. New rules, tailor made for AI firms, may have given these companies an express lane to American's 401K accounts, but that doesn't mean the average person gets to dip into the wealth being showered on insiders.

The big problem with circular financing is that it's little more than a hyper scale game of musical chairs. So far, investors have kept sliding new seats into place each time AI companies have warned that their cash-burning engines were running short (or started screaming "we can't let China win," whatever that means), but there is no guarantee that this will continue.

In fact, it absolutely can't continue indefinitely.

AI-related companies currently make up 45% of the total value of the S&P Index. That's a 20% increase in just two years. No industry have ever come close to this holding this large a share of the market's total wealth. Over those same two years, just five companies—Nvidia, Microsoft, Apple, Alphabet, and Amazon—are responsible for almost all stock market gains. The entire S&P index would be 25% lower were it not for the ungodly valuations assigned to AI companies.

So what happens if, the next time Anthropic or OpenAI hold out their hands, investors can't come up with the funds to keep the music playing?

Well, it would not be good. Ordinary investors may have not been invited to the feast enjoyed by AI billionaires, but they would certainly feel the kick in the 401K if these companies go down.

Some analysts have predicted that, because much of the AI debt exists in the form of equity-finance rather than the sort of leveraged debt behind the 2008 mortgage crisis a meltdown of the AI industry wouldn't result in an economy-wide depression. Cool. That doesn't mean the losses would not be large and sustained. Among other things, it would likely sink large sectors of non-AI technology investment for a decade and gut some of the biggest tech firms that had been successful in other areas before AI can along to be the hot-girl-in-a-red-dress.

In addition to the stock market damage, an AI bubble collapse would leave the nation dotted with dead data centers. That would include projects in the works that could radically affect local and regional electrical grids and water supplies. God help those who had given these companies wads of cash on the promise of jobs, jobs, jobs.

Non AI-companies that had integrated AI into processes from payroll to product design might also find themselves scrambling to find some of those ... what were they called? Oh, yeah. Human workers.

Could we all go back to life the way it was before ChatGPT offered to hallucinate our term papers? Yes. One-thousand times, yes. But it would not be painless.

This may not be the worst outcome, but it is the most likely. AI continues to work ... sort of. It doesn't provide medical marvels. It doesn't solve cheap fusion. It just keeps gradually getting better at faking celebrity images and making up reasonable-sounding legal cases that never happened. Meanwhile, your insurance company keeps depending on AIs with a 90% error rate to determine if you're worthy of getting health care.

AI Meh doesn't put everyone out of work, but it does put a lot of people out of work. (Because corporations that are investing in AI are doing it for that purpose and not firing a bunch of people is a bigger admission of defeat than the average CEO is willing to shoulder when they step in front of the board.)

Not every AI company fails, but some of them do, delivering an economic effect that's a muted version of the bubble popping. Investors stop being willing to prop up every company that slaps AI into the prospectus and start lining up behind a smaller number of perceived "winners."

What kind of disaster is this? The same kind that we're already experiencing. One that continues to accelerate the wealth-concentration engine, widens the wedge between the top 0.001% and everyone else, and powers every other catastrophe that follows in this downbeat list.

This one also comes with the bonus of continuing the economic, infrastructure, and climate disaster that is massive data centers.

In 2003, Swedish / British philosopher Nick Bostrom added an unsettling twist to "simulation theory" — the idea that everything we can see, our entire universe, is actually a program running on some vastly powerful computer. Bostrom argued that, assuming that it's possible for any civilization to reach the technological level where they can generate such a simulation, then the odds that we are in such a simulation are "very close to one".

The reason for this, Bostrom conjectured, is that once a civilization reaches the level of being able to simulate a universe, there's nothing to stop it from simulating another. And another. Ad infinitum.

While I really enjoy strolling around the worlds of No Man's Sky, we clearly are not currently at the fully simulate a universe level. However, we are capable of making AI bots and agents who can simulate human presence on the internet effectively enough to pass multiple versions of the Turing Test and rage bait you into a social media rabbit hole. We can make a lot of those agents.

Despite tools that are supposed to scrape away the digital barnacles on a regular basis, studies showed that 10% to 15% of social media accounts were bots by the early 2020s. With the introduction of AI tools, things are have only gotten worse. On TikTok, bots have sometimes accounted for 97% of new activity.

Earlier this year, digital security firms announced that automated traffic from bots and agents had exceeded human traffic across the entire internet. Since then, both the ease of creation and the sophistication of bots has increased. Of the billions of bots now pushing into every niche, most did not have their origin with a human being. They were created by other bots; often several layers of bots leading back to a master bot that may finally be taking commands from a human.

Bot numbers were swelling and forcing systems to adapt even before AI tools became available to the public. In early days, these bots were easily detected. Many were little more than automated versions of familiar scams whose language was broken and filled with easy to spot phrases. Those kinds of bots are rapidly disappearing. Increasingly, sites are being overrun by AI-powered chatbots that are capable of imitating humans well enough to fool even usually savvy system users.

Every time a new AI model is introduced, it eliminates some of the markers that allowed AI-based text, images, and video to be easily detected or blocked. Bots get better. Scams get better. Filters and detection tools let more bots slip by long enough for them to establish relationships with human users.

Whether it's selling health insurance, roof repairs, or fake vitamins, anyone who has worked in a high pressure boiler room knows that the secret to raking in cash is simply volume. Every day, bots are becoming more numerous. Every day, they are becoming more sophisticated. Every day, they are becoming more capable of using the same tools as retailers to sieve your personal information for a weakness. Or a desire.

If there's one thing that massive data centers are enabling, it's massive number of AI-powered agents. Which are making other agents. Which are trolling for humans. Now that we've entered the "agentic age," most of the tools that websites have built to detect bots are worthless — as are most of the skills you've acquired to fend off Nigerian princes.

It's altogether thinkable that non-human traffic could so overwhelm every aspect of online life that finding another actual human on the internet becomes like hunting for a needle in a stack of fake needles. At the same time they've overwhelming social media, agents are also becoming more adept at building fake web sites, spreading fake news, and stage-managing fads that threaten the health of millions.

Using the internet has always involved an element of risk. Over the last two decades, organizations and individuals have carefully constructed protocols and processes meant to minimize the chance that websites could be easily hacked or taken down using simple forms of attack. These high walls are what makes financial transactions. They mean you can buy a pineapple pizza or bean burrito with fair confidence that a driver will soon deliver a your unhealthy midnight crave to your door.

AI bots threaten all of that.

It's not a perfect measure, but the electricity involved in operating the internet, including all its sites and services, currently runs around 485 terawatt-hours of electricity each year. That's between 1.5% and 2.0% of the total global output. About one third of that currently goes to AI data centers. It won't surprise anyone to learn that this fraction is growing rapidly.

Considering how much of the internet is actually relatively "dumb" infrastructure, we may have already crossed the point where more compute cycles are dedicated to AI than to the thing that makes AI's utility possible. This is unlikely to be a sustainable situation.

The internet may not become completely unusable. It may not collapse under the weight of billions of bots and agents. But even if it doesn't, it could become a much more threatening, more frustrating, more expensive place to visit than the current ad and troll strewn digital wasteland that we love.

Whether it's a more sophisticated version of a "denial of service" attack where websites are buried under spurious requests, an expansion of database breaches, or complete fake websites acting like fiscal Venus fly traps, it could easily become highly inadvisable to conduct any business over the internet. It could also become nearly impossible to determine if the person you're chatting with is a person, much less the specific person you were hoping to contact.

AI doesn't have to launch some kind of coup to take out the internet. It only has to keep growing. It only needs to be Meh.

But hey, you don't have to deal with this crap. All you have to do is tell your AI agent to handle your email and give it a credit card so it can shop for you and pay all your bills. I'm sure it will be fine.

Boom. Bop-a-doom. Bop-a-doom. Bop-a-doom.

Finally! We come at last to something that could outright kill you rather than just leaving you broke, isolated, and at the mercy of sadistic technofeudalist overlords. This threat might not be the most likely, but it is the most likely to test the boundary between merely civilization ending and sustainable gene pool.

Consider this.

Donald Trump is a violence-loving idiot. Vladimir Putin is a violence-loving idiot. Benjamin Netanyahu is a violence-loving idiot. Kim Jung Un is a violence-loving idiot. What do these bozos have in common besides the intense desire to be worshipped and a deep joy they find only by murdering people? They all have their fingers on big red nuclear buttons.

(Don't bother pretending, Benny. We know. We all know.)

For over eighty years, someone has owned a big red button. An amazing number of those people have been stupid and/or violent ... though, granted, not so stupid or violent as the current batch. And yet, no one has pressed that button since way back in 1945.

The biggest reason for this was worked out decades ago under history's most accurate acronym, MAD. Every one of them knows that they could slaughter tens or hundreds of thousands on a whim. But they retain a substantial fear that the outcome would be bad for them, personally. Which is all that matters.

However, are two ways in which AI can be extremely helpful in bringing on a round of mass destruction for all. The first, and possibly largest issue, is that people often display far too much trust to what they're being told by AI, even if what they're being told is flatly wrong. Just the way models can word responses with apparent certainty and a level of writing that mimics a high level of education means that AI has AG. Artificial Gravitas.

Before we completely scoff at the idea that AI might threaten the sort of nuclear mushroom field scenario that gave every kid of the 1960s collective duck-and-cover PTSD, don't forget that these models already came close to nudging the US toward a war with China and gave a thumbs up to bombing a girls' school. People trusted what came out of flawed models way too much.

There have been several occasions in which the apparently irrational decision of a single human being was all that saved the world from nuclear war. What happens if that person is either out of the loop or facing warnings from an LLM model that's been credited with "super intelligence?" Is that one lonely woman or man still going to be brave enough to say no, even if Claude 10.0 is screaming "Yes! Yes! Yes?"

The second factor that has kept the world from frying through these long post-Trinity years is simply numbers. MAD is a terrible solution. This awful solution has worked for decades. But if only keeps working if there aren't many people with access to the buttons.

It's unlikely that any of the AI models out there today are going to teach you how to make a nuclear weapon, much less how to mine, refine, and enrich roughly 150 pounds of uranium. (I don't know this, because I haven't asked.) However, models currently available to the public have proven adept at designing viral genomes, along with both biological and chemical weapons. They are also frighteningly good at battering their way through digital defenses and spreading false information which might sway people already engaged in big red button management.

The point is that AI threatens to put some form of mass destruction in the reach of many more fingers. Raise the numbers high enough, and eventually one of them belongs to a Dylan Roof or an Aum Shinrikyo cult; people who are much more excited about their ability to inflict damage than they are worried about what kind of return fire they might receive.

AI can make it easier for more people to get their fingers on buttons. It may also make those with buttons more likely to make false decisions based on bad programming or hallucinations that absolutely intrinsic to the design of LLMs.

And ... that's it. It's far from an exhaustive list of the ways that AI can generate huge levels of human misery without the need to develop divine powers or hand over the keys to Skynet.

LLM models are not intelligent, much less super intelligent. But they can mimic human behavior at a level that makes them appear to reason. Badly. Win, lose, or meh, they can still hurt us.

After all, Donald Trump is stupid. Just look at the damage he has done.