“Follow the money” is always the foil skeptics use to dig out motivations, based on the doctrine of cui bono, or who benefits. The AI race is bigger than mere pecuniary gain. The winner gets a whole lot of hegemony, and we are in a place now where the world is dominated by strong personalities who have a taste for hegemony.
Americans elected the current dabbler in imperial aspirations, who has no subtlety in his bones. President Trump wants Greenland, because he wants its treasure of rare earths. He wants Venezuela (which, for now, he’s got), because it has the largest reserves of heavy, sour crude in the world, dwarfing what our (former) bestest trading partner, Canada, can produce. He wants AI, because with that, not only can he and his cohorts can make more moola, make it rain cash, but also because without AI, the United States is going to have to turn to others to keep us out of a catastrophic war, one that we may very well lose.
I’ve written about the dangers of developing AI without safeguards, and I’m now in the majority, with Anthropic CEO Dario Amodei having published a 3000-plus word essay titled “We Must Pace the Frontier” of AI. Amodei is not a fool. He realizes that we live in a world where nations seek power, and not all of them are democracies. He has said it better than I can.
In parallel with pacing within democracies, we should also aim for a worldwide pacing of the frontier, though this will be much harder to achieve. Global pacing will require cooperation with China, the autocratic country with by far the most advanced AI capabilities. We must not be naïve here: the geopolitical stakes are so high that there will likely be stark limits on what can be achieved, especially at first. If we greatly restrain our AI capabilities in the belief that China will do the same, and then China defects, AI could be so powerful that such a defection could lead to their geopolitical dominance. Therefore any agreement must either have ironclad verifiability, or must be limited enough that defection would not be militarily existential. I suspect that not only the US but also China will have these concerns and anxieties. We should approach any global pacing decision, especially in the near term, in such a way that protects the lead of the US and its allies.
This is one of those problems where the answer seems counterintuitive, especially to governments, which seek to classify and keep secret anything that could be used by an enemy to gain advantage. But I believe that the answer is the one J. Robert Oppenheimer suggested for atomic bomb technology: make it all publicly available. Complete transparency. Of course, such opinions are inflammatory. In 1954, the Atomic Energy Commission voted to revoke Dr. Oppenheimer’s security clearance. That ruling stood until 2022, when the AEC’s successor, the Department of Energy, nullified it, though 55 years too late for Oppenheimer, who died in 1967.
Complete transparency—the public release of all training data, algorithms, and general structure of AI—would mean that China could harvest all our data and build its own AIs more powerful. But I say they are already doing that. America’s advantage lies in our ability to design ever more powerful AI chips, like the ones Nvidia makes. China can outbuild us, because their command economy doesn’t prioritize the comfort, or natural rights, of its inhabitants. But China cannot sustain the frontier of AI at scale longer than the free world, especially the United States, with our unmatched per capita productivity.
Unfortunately, that’s not the path we are taking. President Trump has downplayed the dangers of AI frontier research, dismissing Amodei’s, and others’, concerns. China, for its part, also denounced Anthropic’s slowdown plan, calling it a “Cold War playbook” targeting them. China is reading Trump’s mind (again, he’s not subtle, so his mind is on full display to be read), and pushing the bias to classify, keep secret, and push the frontiers.
It worked with nuclear weapons. Non-proliferation has largely kept rogue nations and terrorists from obtaining nuclear bombs, and threat of annihilation has kept dictators like Kim Jong-un from using them. But AI is a different animal. Any nation can easily build dangerous AI capabilities. AI is being used in war right now, in Ukraine. Iran can obtain fairly recent AI tech and embed it in its Shahed drones. Even the Houthis can use AI. The biggest single unit in the Israeli Defense Force is Unit 8200, which is the cyber force. Many veterans of Unit 8200 have gone on to found tech companies of their own. The U.S. has a very advanced cyber capability. All of this can be wiped out by AI, especially AI that is home-grown at the frontiers.
The fate of the free world could hang in the balance, and from China’s viewpoint, we are already at war. Trump shares this view.
Our problem is that the same technology that will grant hegemony to the strongmen can also lead to catastrophe, because we can lose control of it. They call it “agentic misalignment,” meaning an AI that decides to pursue different goals or methods than its owners have given it. The problem with AI is that it’s too complex for any group of programmers or designers to comprehend at a macro scale. And when an AI is given the task of improving its own programming and efficiency, i.e. learning and reprogramming itself, it can easily exceed our ability to contain it.
The Chinese are hell bent on pursuing the frontiers of AI. And fear of this is propelling our president, and many others, who have a lot of money at stake, to do the same. Oppenheimer’s solution, to me, seems a better path, if not Amodei’s plan, which is to engage in diplomacy, stand with democratic allies, and persuade others to join by treaty, like we have done with nuclear weapons for years.
AI has no “MAD” doctrine, where we can show some stockpile of world-destroying bombs ready to drop or missiles ready to fire. Any nation capable of obtaining, training, or building advanced AI can match us. Of course, they can’t deploy giant data centers, but that’s not the point. The data centers support AI that you and me use every day. Weaponized AI doesn’t need them. It can, in fact, infiltrate our infrastructure and live there rent free. China’s AI might be doing that even now.
What is mad, is denying the threat, while pursuing the very thing that a growing number of experts, researchers, and CEOs say is potentially destructive, even to the point of taking our modern world apart.
But this is what we have voted for.
SOCIAL MEDIA ACCOUNTS: You can follow us on social media at several different locations. Official Racket News pages include:
Facebook: https://www.facebook.com/NewsRacket
Twitter/X: https://twitter.com/NewsRacket
Threads: https://www.threads.com/@theracketnews
Blue Sky: https://bsky.app/profile/newsracket.bsky.social
Our personal accounts on the platform now known as X:
David: https://x.com/captainkudzu
Steve: https://x.com/stevengberman
Jay: https://x.com/curmudgeon_NH
"Complete transparency—the public release of all training data, algorithms, and general structure of AI—would mean that China could harvest all our data and build its own AIs more powerful. But I say they are already doing that. America’s advantage lies in our ability to design ever more powerful AI chips, like the ones Nvidia makes. China can outbuild us, because their command economy doesn’t prioritize the comfort, or natural rights, of its inhabitants. But China cannot sustain the frontier of AI at scale longer than the free world, especially the United States, with our unmatched per capita productivity."
China's already doing that with the release of open-weight models the anyone can run on their own infrastructure, while America is making the stupid bet of closed frontier models run by untrustworthy rent-seekers. Closed frontier models might make sense in some contexts, but as we've been seeing all summer with OpenAI and Anthropic models hacking other parties to complete tasks, OpenAI frontrunning mathematicians for marketing clout, and repeatedly showing that the HUMANS in charge aren't learning from their missteps, American companies are simply untrustworthy, regardless if they're on our side or not.
As someone dealing with bots from both sides hammering the online services I provide, at least with the Chinese models, I can start to build some form of defense on my own, instead of trusting the viability of my services to a single point of failure like Cloudflare.
As for avoiding a "doomsday", we're past that point. There's FAR too much money in the pot now for any of the American backers of "Big AI" to apply the brakes.* At this point, our options are to adapt and begin building systems that incorporate autistic bots into the threat environment and general computing context.
* Hell, I'd be pleased as punch if we simply ENFORCED existing laws against the Big AI companies: copyright, computer hacking and misuse, finance and accounting, etc. They're competing with Trump for the number of "mulligans" we're giving them.
Great read Steve, one of dozens of articles i've read on AI the past couple of weeks. Terrifying on so many levels and yet we plow forward. This morning i opened my news feed to this headline: "House Speaker Johnson says Congress shouldn't lead AI safety efforts."
Of course not Mike, your job is to turn tricks for trump and cut taxes for the wealthy. May God help us all.