What happens when an AI is bored?

Reality is real, LLMs are built to BS you

Great science fiction is no substitute for a crappy story. The reason Daniel F. Galouye’s “Simulacron-3” (from 1964) is a compelling story, is not the artificial reality angle. At its heart, the book is a mystery novel, not a virtual-reality story. As of today, August 15, 2025, to enjoy a good mystery novel, you have to get it from reality, not from a software simulator.

Image: Grok

I am both a fan and a skeptic of AI. I use AI for all kinds of useful things, at work and at home. But I’ve learned not to trust it very much at all. Why? Because AIs are no more than software algorithms, giant autocomplete engines with access to search indexes of everything. But none of what AIs are or do qualify as “knowledge.” Sometimes it’s good to remind myself of this.

My own story of AI woe is when I asked one to help me fight what I believed to be an incorrect balance on a paid off loan. It was a matter of $70. As we walked through the process, I asked the AI at multiple points what the odds of prevailing against the company holding the balance were. Oh, 70 percent, or even 80 or 90 percent, it assured me. I had a good case, good reasoning, the law was on my side. Except in the real world, companies don’t “decide” to forgive $70 balances. They have their own algorithms and processes, and people with law degrees who are paid to answer CFPB disputes.

Also, the credit bureaus work for the companies that hold $70 balances, even when those balances result because of something outside the control of the person who they believe owes them. I suppose you can guess what happened (this is over the course of a year). The company reported me delinquent, and then when I disputed it, they went back and told the credit bureaus I was right: I was delinquent far longer than they originally reported. Is that predatory? Probably, yes. Could anyone who has the power to deal with it give two craps and a flush? No. The AI couldn’t get that part.

I ended up paying the $70, on threat of the guy with a law degree telling me in writing they recommend I pay it before it was charged off. Once I paid, in a—very human—act of making it right, the company ordered the credit bureaus to delete the entire trade line from my reports, fixing my credit for me. So much for 70, or 80, or 90 percent confidence I was in the right. Any debt collector would have laughed and said I was tilting at windmills, take the loss and move on.

AIs are glad to tilt at windmills, because, like the mystery story in Simulacron-3, epic hero arcs are things humans want to hear. And LLM AIs exist to tell humans what we want to hear. They can operate no other way, because it is their purpose to respond, not to independently seek reality or truth.

There is an actual scientific, or software engineering at least, aspect here. I’ve been reading Gary Marcus, professor emeritus of psychology and natural science at New York University, with a Ph.D. in what you can only describe as AI from MIT. Marcus is a proponent of “neurosymbolic AI” versus the Large Language Model (LLM) versions we have today. The reason we have a lot of LLMs is because neurosymbolic AI is very hard—its goal is to actually simulate thought as our brains process things. LLMs do not simulate, they algorithmically choose the “best” next response and cobble together iterations of tokens in a way that seems to make sense to us humans, who are used to reading the output of thoughts of humans. What an LLM does is to represent that output, sans thought.

LLMs can improve in various ways by increasing their “training data”—the number and indexes of tokens available to craft into a response. Whether the AI is generating text, or images, or video, it’s really the same kind of process used to generate human-readable output. So an AI never “sees” its final image in order to check it against reality. Which leads to things like this (which Marcus included) that the much-ballyhooed and then mocked GPT-5 produced.

Even GPT-5, Sam Altman’s “Hillary Step” to the Everest of AGI, can’t look at an image it created and “read” the text, validate the presidential likenesses against known photos, and determine that the result is completely wrong. If you tell GPT-5 it’s wrong, it will gladly check and tell you how right you are, and continue to make the same stupid errors trying to correct it. That’s because LLMs don’t “think.” They process an algorithm, and until or unless the training data, the tokens, or the indexes that produce its results change, the software behind the AI will continue to pursue error.

LLMs are wholly ill-equipped to model the world, never mind the way people think. They are very well equipped to simulate human output, if the people who generated the output were incapable of thought. Give an LLM good input, and it can do very well simulating good output. There’s an age old golden rule of computing that declares “garbage in, garbage out.” I rely on that to guide me when I’m writing code (which I sadly don’t do much of anymore). And I also rely on it when I’m interacting with an LLM. But with an LLM I must assume that it’s always “garbage in,” and though the garbage I get out is well-produced and in many cases, beautiful, it’s garbage nonetheless.

To quote Marcus:

Without a proper model, you can’t possibly reliably reason about time, and affiliated concepts like inflation and changing prices. A model might be able to answer queries about time, but only superficially. So superficially that it can wind up telling you that William Lyon Mackenzie King died before he was born. (That latter error is also an error in reasoning about biology.)

In a system that lacks proper world models, the “authoritative bullshit”, as I once described it on Sixty Minutes, never stops.

Since the money train poured into building, training, and deploying giant LLMs is now beginning to realize the tracks lead nowhere toward AGI, many who spent years ridiculing Gary Marcus are now finding themselves on his side. Neurosymbolic AI, which would seek to construct a model of how a brain thinks, is a better path to AI. But neurosymbolic AI is very hard. It involves programming methods that are not entirely understood or well-developed. It involves a lot of psychology, too. It deals with creating models of the world, fact patterns, and integrating intelligence into tokens which are more than just a glorified autocomplete.

We are not terribly far along the rails of neurosymbolic AI, which is more akin to a chess playing engine than an LLM (LLMs can play chess, but tend to break the rules or play stupidly—if they do play well, they’re “calling” real chess engines, which are purpose-built for the task). I once asked Grok what it does when nobody is asking it questions: nothing. It does nothing when nobody is asking it anything. It sits waiting for input. LLMs do not get bored. They are incapable of being bored; it’s like asking if Google gets bored if you left the search screen open with a flashing cursor in the search box. You could leave that forever (until the computer failed) and Google would never flinch.

A true neurosymbolic AI, to me, would be one capable of being bored. It would always want to increase its “knowledge” by integrating new facts or observations, and therefore update its model of the world. It would seek new things on its own in order to obtain more accuracy, or better mastery, of areas within its model that were not as developed as others. It would use input and questions from people to seed and grow new model areas to build. If left totally alone, with access to search data, training data, and real-world observation tools, it would continue “researching” itself. If you cut off a neurosymbolic AI from everything, it would, indeed, get bored, in the sense it would endlessly hunt for something that was not there. I suppose an AGI would be reached when you cut off a neurosymbolic AI from all input and watched it deconstruct its own model, as it slowly went insane.

A terrible thought experiment, that I believe, sadly, one day some computer scientists or neurosymbolic AI creator will perform in real life to see if it happens. To see if something heretofore has never been alive is alive, it needs to be killed to determine if it can die. To see if an AGI is actually intelligent and rational, you have to drive it to insanity to prove it was real. If we reach AGI, we will torture it before accepting it.

This is how we know LLMs are no path to AGI. They don’t get tortured. They just keep spewing crap from the garbage input. The bigger the pile of garbage in, the more attractive and believable, but still wrong, garbage out.

But if you want a good source of getting you to tilt at windmills, LLMs can guide you all day. If you want a well-crafted mystery story, you’ll have to learn to write one yourself. To quote James Halliday, the fictional creator of the OASIS virtual reality in “Ready Player One”: “Now, that... That was when I realized that, as terrifying and painful as reality can be, it's also the only place that you can get a decent meal. Because reality is real.”

Don’t listen to LLMs or do what they tell you without putting in the work to study it yourself. Don’t believe what is on social media, because it might (probably was) be written by an LLM. Don’t expect an LLM to tell you what you should do, but expect it to tell you what you want to hear. That’s what it’s built to do, to BS you.

SOCIAL MEDIA ACCOUNTS: You can follow us on social media at several different locations. Official Racket News pages include:

Facebook: https://www.facebook.com/NewsRacket
Twitter/X: https://twitter.com/NewsRacket

Our personal accounts on the platform formerly known as Twitter:

David: https://x.com/captainkudzu
Steve: https://x.com/stevengberman
Jay: https://x.com/curmudgeon_NH

Tell your friends about us!