JEconomist Bryan Caplan was sure that the artificial intelligence integrated into ChatGPT was not as intelligent as claimed. The question: Could AI pass the 2022 midterm exam of its undergraduate class?
Caplan, of George Mason University in Virginia, seemed well placed to judge. He’s made a name for himself placing bets on a range of newsworthy topics, from Donald Trump’s 2016 electoral chances to future US college attendance rates. And he almost always winsoften betting against predictions he considers hyperbolic.
Such was the case with wild claims about ChatGPT, the AI chatbot that has become a global phenomenon. But in this case, it sounds like Caplan – a libertarian professor whose arguments range from calls for open borders For criticism of feminist thought – will lose his bet.
After the original ChatGPT got a D on its test, it wagered that “no AI could get an A on 5 of my 6 exams by January 2029.” But, “to my surprise and no great dismay,” he wrote. on his blog, the new version of the system, GPT-4, earned an A a few months later, scoring 73/100, which, if he had been a student, would have been the fourth highest score in the class. Considering the astonishing rapidity of improvement, Caplan says his chances of winning seem slim.
So, is the hype justified this time around? The Guardian spoke to Caplan about what the future of AI might look like and how he’s become an avid bettor.
The conversation has been edited and condensed for clarity.
You bet no RN could get an A on five of your six exams by January 2029 – and now one has passed. How much did you bet?
I tried for 500 dollars. I think it’s a reasonable prediction that I will lose the bet at this point. I just hope to be lucky.
So what do you think this means for the future of AI? Should we be excited or worried or both?
I would say excited, overall. All progress is bad for someone. Vaccines are bad for funeral homes. The general rule is that anything that increases human production is good for human living standards. Some people lose, but if you said we only want progress that benefits everyone, then there would be no progress.
I have another bet on AI with Eliezer Yudkowsky – he’s the biggest and probably the most extreme AI pessimist, in the sense that he thinks it’s going to work and then it’s going to wipe us out. So I have a bet with him that because of AI, we’ll be wiped off the face of the earth by January 1, 2030. And if you’re wondering how could you have a bet like that, then that you are one of the people who are going to be wiped out – the answer is that I just prepaid it. I just gave him the money up front and if the world doesn’t stop he owes me.
How could we theoretically be annihilated?
What I consider a bizarre argument [more broadly] is that once the AI becomes intelligent enough to increase its own intelligence, it will switch to infinite intelligence in an instant and that will be it for us. [That view is endorsed by] very intelligent and eloquent people. They don’t look crazy, but I just think they are.
They kind of talked themselves into a corner. You start with this definition of: imagine there is an infinitely intelligent AI. How can we stop him from doing what he wanted? Well, once you just put it that way, we couldn’t. But why should you think this thing will exist? Nothing else has ever been infinite. Why would there ever be something infinite?
What goes into your thinking when you decide: is it worth a bet?
The kind of bets that pique my interest are those where someone just seems to be making over the top hyperbolic claims, claiming to have a lot more faith in the future than I realize. So far it has served me perfectly. I had 23 bets that materialized; I won all 23.
I’ve had several other instances of people telling me how awesome the AI is, then I checked out for myself and they were clearly exaggerating a lot. And so I just thought the exaggeration was continuous, and sometimes you’re wrong. Sometimes someone says something that seems ridiculously over the top and that’s just the way they say it.
In other words, you tend to reject the most dramatic results possible.
I almost always bet against the drama. Because it appeals to the human psyche to say exciting things, and my point is that the world generally isn’t that exciting, actually. The world generally continues to be as it was. “The best predictor of the future is the past” is a saying that I find so wise, undeniable. If someone doesn’t take it seriously, then I have a hard time taking it seriously.
So if you lose the AI bet, is that an indicator that the hyperbole is justified?
I think this shows for this particular case that GPT-4 has advanced much faster than expected. I think this means that the economic effects will be much larger sooner than expected. As I was expecting very little effect, it might be 10 times bigger than I thought and not be huge. But certainly on this issue, as if I had rethought my point of view.
The only story I could think of that would redeem my original skepticism would be if they just added my blog post to the training data and then pretty much spit out my own answers at me. But here’s the problem: I actually have a new position where I gave GPT-4 a totally new test that I’ve never talked about on the internet, and it got the highest score, so I think it’s genuine.
And what happens next?
There is a general rule that even when a technology looks great, it usually takes much longer to have significant economic effects than expected.
The first telephones were in 1870; it takes about 80 years before this technology even gives us reliable phone calls to Europe. Electricity seemed to have taken several decades to be widely adopted, and the internet also seemed to have taken longer than expected.
I remember several years when rollback didn’t work on email. I don’t know how old you are, but I remember when you couldn’t back up an email. And it went on for years like that. You might think this would be solved in three minutes. But any time humans are involved in technology adoption, there are just a bunch of different issues, different hiccups. So as to whether GPT is really going to transform the economy in a few years, I would consider that to be pretty amazing. It’s almost unprecedented.