OpenAI chief says era of giant AI models is coming to an end; a GPU crisis could be one of the reasons why

The era of ever greater artificial intelligence models are coming to an end, according to OpenAI CEO Sam Altman, as cost constraints and diminishing returns are holding back the relentless scaling that has defined progress in the field.

Speaking at an MIT event last week, Altman suggested that further progress would not come from “giant, giant models”. According to a recent Wired report, he said: “I think we are at the end of the era where it will be these giant, giant models. We will make them better in other ways.

Although Mr. Altman did not quote him directly, one of the main driving forces behind the pivot of “scaling is all you needis the exorbitant and unsustainable cost of training and running the powerful graphical processes needed for large language models (LLMs). ChatGPT, for example, would have been required more that 10,000 GPUs to train, and requires even more resources to operate continuously.

Nvidia dominates the GPU market, with around 88% market share, according Search John Peddie. Nvidia last H100 GPUsdesigned specifically for AI and high performance computing (HPC), can cost up to $30,603 per unit – and even more on eBay.

Training a state-of-the-art LLM can take hundreds of millions of computational dollars, said Ronen Dar, co-founder and chief technology officer of Run AIa compute orchestration platform that accelerates data science initiatives by pooling GPUs.

As costs skyrocketed while benefits leveled off, economies of scale turned against ever larger models. Rather, progress will come from improving model architectures, improving data efficiency, and advancing algorithmic techniques beyond the cut-and-paste scale. The era of unlimited data, computing, and model size that has remade AI for the past decade is finally coming to an end.

“Everyone and their dog buys GPUs”

In a recent interview on Twitter Spaces, Elon Musk recently confirmed that his companies Tesla and Twitter were buying thousands of GPUs to develop a new AI company that is now officially called X.ai.

“It seems like everyone and their dog is buying GPUs at this point,” Musk said. “Twitter and Tesla definitely buy GPUs.”

Dar pointed out, however, that these GPUs might not be available on demand. Even for hyperscaler cloud providers like Microsoft, Google, and Amazon, it can sometimes take months — so companies actually reserve access to GPUs for themselves. “Elon Musk will have to wait to get his 10,000 GPUs,” he said.

VentureBeat contacted Nvidia for comment on Elon Musk’s latest GPU purchase, but did not receive a response.

Not just on GPUs

Not everyone agrees that a GPU crisis is at the heart of Altman’s comments. “I think it’s actually rooted in a technical observation over the last year that we may have made larger models than necessary,” said Aidan Gomez, co-founder and CEO of Joinwhich competes with OpenAI in the LLM space.

A TechCrunch article reporting on the MIT event reported that Altman considers size a “false measure of model quality”.

“I think there’s been way too much focus on the number of parameters, maybe the number of parameters will increase for sure. But it reminds me a lot of the gigahertz race in chips in the 1990s and 2000s, where everyone was trying to point to a big number,” Altman said.

Yet the fact that Elon Musk just bought 10,000 Datacenter-grade GPUs means that, for now, access to GPUs is everything. And since that access is so expensive and hard to come by, it’s certainly a crisis for all but the wealthiest AI-focused companies. And even OpenAI’s pockets are only so deep. Even they, it turns out, might ultimately have to look in a new direction.