How ChatGPT makes the Statue of Liberty disappear
Something to keep in mind amidst all the "AI" hype.
“Write about it if I like or anything if I like but not there, there is no there there.”—Gertrude Stein, Everybody’s Autobiography
There’s a debate happening about ChatGPT. On one side of the debate you have tech bros and /r/philosophy users, and on the other side are actual sentient humans. The debate usually goes like this:
Person: ChatGPT doesn’t understand what’s it’s saying, it’s only putting words into sentences that sound good to us.
Tech bro: Isn’t ‘putting words into sentences’ exactly what you’re doing?🤯🤯🤯
The companies who make generative AI software love this debate because it makes them look mysterious and godlike. In this essay I hope to convince you that both of the debaters are wrong (mostly the tech bro), but not for the exact reasons they think. Yes, when humans talk they are “putting words into sentences”, and yes, ChatGPT is making things “that sound good to us”. But those are not the same thing. How much that distinction will matter in the grand scheme of things is yet to be seen, but I do think it is important to pierce the hype bubble early, before this software becomes trusted more and more.
Understanding vs Persuasion
ChatGPT is software (a “Large language model”, or “LLM”) that generates text which appears very human like. It’s almost like magic. Actually- it is exactly like magic if we’re talking about the Siegfried & Roy, variety. To create the illusion of humanity the ChatGPT software is doing some interesting sleight-of-hand. And just like actual sleight-of-hand tricks, in order to be fooled- the audience has to view it from a very specific perspective.
Here’s a famous example of what I mean by “specific perspective”- In 1983, illusionist David Copperfield made the Statue of Liberty “disappear”. It was this big deal television event. There was a live audience in front of a big stage with a big curtain. Behind the stage was ol’ Lady Lib, brightly lit up with floodlights. At the big moment, the curtain closes and abra-cadabra, it re-opens and the statue is gone! Here’s a video of the event if you’re under a million years old and don’t remember it:
How this trick worked is actually really clever and simple: While the curtain was up, Copperfield turned the statue’s lights off, and then rotated his whole setup —stage, curtain and live audience— a few degrees to the side. When the curtain came back down, it was no longer facing the the statue (who’s lights were now off). ✨Magic!✨ To further sell the illusion, a bunch of live audience members are interviewed who are justifiably astonished at what they just saw.
But dear reader, the live audience was also part of the illusion!1 The actual audience was the viewers at home across America, watching via the narrow perspective of the live TV broadcast.
This was an illusion that could only have worked during the age of broadcast TV, when all the conditions could be tightly controlled. If Mr. Copperfield tried it today, someone would surely be flying an unauthorized drone over the whole setup and live-streaming the behind-the-scenes feed to Facebook. Overall, the event was clever, entertaining, and fun. But everyone understood it to be just that- a fun, clever trick that only “worked” in specific circumstances. To my knowledge, nobody wrote articles about how Mr. Copperfield’s new “data-driven startup” was going to “shake up the world of statue disappearing”.
So at what “specific perspective” is ChatGPT targeting its illusion? The perspective of a human brain on a computer. Our experience of reality is determined by our senses and limited “wetware”, and ChatGPT is like David Copperfield, not actually making statues disappear —not actually understanding what it’s saying— but outmaneuvering our ability to determine otherwise.
I often think about how a lot of our tech would likely be weird and worthless to aliens who are built even slightly different than us. For example, consider how a television set (like the ones needed for the Statue of Liberty trick) produces only three colors of light: red, green, and blue. This is because our eyeballs detect reddish, greenish and blueish light. Our minds create all the other colors by “mixing” those three. So a TV doesn't need to produce (for example) orange light, it just needs to tickle our rods n' cones the same way something actually orange would. It's why you'll sometimes hear that purple light “doesn't exist”. “Purple” is what our minds make up when blue and red are mixed. But the light between blue and red on the EM spectrum is well, not purple.
An alien who’s eyeballs and brain don’t work exactly like ours do would probably not be able to use a TV (or laptop, smartphone, etc). A TV is not designed to re-create the real world, it’s designed to trick human brains into re-creating it2.
ChatGPT is only “using words to make sentences” inasmuch as your TV set is using copper cladding to make Statues of Liberty.
Bypassing your bullshit filters
Generative image software like Midjourney or Dall-E (that create jpegs from a text prompt) work on a similar principle as ChatGPT and can serve as another way to understand. “Inside” Midjourney there are no 3D models of the world, no concepts or ideas. What these programs are doing is not making a representation of an actual thing, they are generating the inputs our brains will recognize as the thing. It sounds similar, but understanding the difference will help explain what I feel are the impassable limitations of this kind of “AI” technology. Here’s an example:
A recent New York Times article featured several generated images of “movies that don’t exist”. It’s a cool demo, but I want you to look at this one specific image:
It’s a person on a futuristic motorcycle, right? But, look harder. Try to find the motorcycle. There’s a circular thing that’s sort of wheel-ish, and there’s a blob sticking out of the circular bit that’s kind of where a handle might be. But the person’s “body” just kinda melts away, and where you might expect the back wheel to be is a just blur of black spaghetti. There’s stuff here, but there’s no actual motorcycle (not even a futuristic one). But when the image first popped up on the screen, we all saw a sci-fi person on a sci-fi motorcycle, right?! So what gives?
There is no motorcycle because the AI doesn’t understand what a motorcycle is, let alone how to draw one. What it does understand is where to align pixels so that human brains will quickly infer a motorcycle. To illustrate the difference, compare the above image to to this bad drawing I just made:
My (objectively great) drawing is a representation of the underlying structure of a motorcycle as I understand it. By that I mean it has motorcycle parts, like wheels, a motor, etc. and they are basically connected in the way I’m pretty sure that motorcycle parts are. What this (again, objectively great) drawing demonstrates is that I have an understanding of what “motorcycle” means, even if my drawing doesn’t fool you into believing you’re looking at one. Children inherently draw this way too. It’s why a hand drawn by a child looks nothing like a hand but will probably be attached to a body and have the correct number of fingers. A child gets what “hand” means beyond it's superficiality.
The images Midjourney generates have no underlying structure —no actual motorcycle— because it wasn’t programmed to draw “its interpretation of a motorcycle”, it was programmed to persuade a human they’re looking at one. That’s how it can get away with the back wheel being a bunch of spaghetti and yet you still go “oh, hey 👉motorcycle👉” when you see it. My drawing conveys the concept of a motorcycle, Midjourney conveys a perception of one.
One day soon, I’m sure that this software will get so advanced at showing people what they expect to see, that it will generate stuff indistinguishable (to us) from the real McCoy. Already they’re much better at hands and faces; look how far along Midjourney has come in just the past year:
Here’s another image which I think is a good demonstration of what’s (or rather what’s not) going on “inside” these programs:
This image was posted to the /r/Midjourney subreddit with the title “V5 almost getting text”. I’d say it actually appears to be nailing the text, but the letters don’t mean anything. Letters are shapes, and Midjourney kind of “gets” shapes. But it has no idea that shapes can mean anything. Its images are not built off of any internal model of reality, no “story”, of what a protester (or even protesting) is, only what humans are statistically likely to infer as such. It’s creating images from the “top down”, starting with what humans want, and working backwards from there.
Maybe the simplest metaphor to describe what Midjourney et al, are doing is to say it’s like those sidewalk drawings that look 3D when viewed from a certain angle.
ChatGPT is doing with text what Midjourney does with pixels. It’s generating the “shapes” of ideas. Any meaning those shapes might appear to have is an illusion made for us, carefully constructed and targeted at brains that evolved in a place where words are meaningful.
An LLM is not outputting based on what is true, or even based on an internal model of what it believes to be true. It’s outputting what you are most likely to perceive as plausible, which is often, but not always, the same thing. A scarier-sounding way to phrase it is this: GPT’s function is to generate the inputs least likely to set off your brains’ bullshit filters.
I think we are lucky, to an extent, to be able to experience this software in an “unfinished” state, because it gives us insight into how the illusion works. If this stuff was kept under wraps a few more years, I think it would be much harder to have any sort of debate, and you’d probably have some misled people making a more impassioned case for sentience that I’m sure the tech corporations would just love.
Lastly, it’s not my business to make predictions, or worry too much about this still nascent tech. But I do worry a little. Big tech companies and the very online journalists they’ve wowed, are telling us this is “intelligence”, and it likely is in a way- just not one humans can relate to. But those same people have also said that Twitter was a “town square” or that Facebook was about “connecting”, when what they all really wanted was to manipulate us into making them richer. And now they’re trying to out-engineer our own brain’s ability to detect bullshit?
Podcast host Ezra Klein recently put it like this on his show:
We’re making content with no “truth value” or “embedded meaning” much more persuasive [than it is now]. What has, I think, ruined a lot of the internet, is just how much of the content is there, not for you, or to be accurate or even to be enjoyed, but is there to just try to get you to do something that you didn’t even realize anybody was trying to get you to do.
LLMs don’t need to be 100% factual to be useful, just like a self-driving car doesn’t need to never crash to be safer than a human driver. But if we’re going to trust “AI”s to provide us with knowledge, we need to be able to see a reasoning of their outputs. But generative AI can’t do that because they don’t know anything. Kinda like how TVs don’t know what a Statue of Liberty is, but can still fool our brains into seeing one disappear.
I have seen reports that they were fooled too, that blinding lights were shined on them to prevent them from noticing the world around them spinning as the platform they sat on rotated. But I feel like it’s much more likely they were paid $100 to lie and signed an NDA or something?
If this is blowing your mind a little and you want to learn more, YouTube channel “Technology Connections” has a fantastic video on the subject.
Loved this article. It’s astonishing how the ChatGPT conversation tends to skip the fact that the AI has no clue about its input/output. But as you point out, content on the internet is not meant to inform but to persuade us to do something.