AI breakthroughs have been a question of when, not if. Google unveiled long-awaited new details about its large language model Gemini. Claude 3 is arguably now one of the most powerful AI models out there, if not the most powerful.

Preview, if you will, for us ChatGPT 5. I expect it to be a significant leap forward. But what if that core assumption that models can only keep getting bigger and better is now fizzling? Is there really a slowing in progress because that wasn't

expected? It could spell cracks in the Nvidia bull story. We're increasing GPUs at the same rate, but we're not getting the intelligence improvements out of it. Calling into question the gigantic ramp in spending from

Amazon, Google, Microsoft; a rush for tangible use cases and a killer app. I'm Deirdre Bosa with the TechCheck take: Has AI progress peaked? Call it performance anxiety. The growing concern in Silicon Valley is that AI's rapid

progression is losing steam. We've really slowed down in terms of the amount of improvement.  Reached a ceiling and is now slowing down. In the pure model competition, the question is, when do we start seeing an asymptote to scale?

Hitting walls that even the biggest players, from OpenAI to Google, can't seem to overcome? Progress didn't come cheap. Billions of dollars invested to keep pace, banking on the idea that the returns would be outsized, too.

But no gold rush is guaranteed to last, and early signs of struggle are now bubbling up at major AI players. The first indication that things are turning: the lack of progression between models. I expect that the delta between 5 and 4 will be the

same as between 4 and 3. Each new generation of OpenAI's flagship GPT models, the ones that power ChatGPT, they have been exponentially more advanced than the last in terms of their ability to understand, generate, and reason.

But according to reports, that's not happening anymore. There was talk prior to now that these companies were just going to train on bigger and bigger systems. If it's true that it's topped out, that's not going to happen

anymore. OpenAI has led the pack in terms of advancements; its highly anticipated next model, called Orion, was expected to be a groundbreaking system that would represent a generational leap in bringing us closer to AGI, or artificial

general intelligence. But that initial vision is now being scaled back. Employees who have used or tested Orion told The Information that the increase in quality was far smaller than the jump between GPT three and four, and that they believed Orion isn't reliably better than its predecessor at

handling certain tasks like coding. To put it in perspective, remember ChatGPT came out at the end of 2022. So now it's been, you know, close to two years. And so you had, initially, a huge ramp-up in terms of what

All these new models can do. And what's happening now is you've really trained all these models, and so the performance increases are kind of leveling off. The same thing may be happening at other leading AI developers, the startup Anthropic.

It could be hitting roadblocks to improving its most powerful model, the Opus, quietly removing wording from its website that promised a new version of Opus later this year, and sources telling Bloomberg that the model didn't perform better than previous versions

as much as it should, given the size of the model and how costly it was to build and run. These are startups focused on one thing: the development of large language models with billions of dollars in backing from names like Microsoft and Amazon and venture capital.

But even Google, which has enough cash on hand to buy an entire country, may also be seeing progress plateau. The current generation of LLMs are roughly—a few companies have converged at the top, but I think we're all

working on our next versions too. I think the progress is going to get harder. When I look at 2025, the low-hanging fruit is gone. You know, the curve—the hill is steeper. Its principal AI model, Gemini, is already playing

catch-up to OpenAI and Anthropic. Now, Bloomberg reports, quoting sources, that an upcoming version is not living up to internal expectations. It has to make you think, okay, are we going to go through a period here where we're going to need to digest?

All this hundreds of billions of dollars we've spent on AI over the last couple of years? Especially if revenue forecasts are getting cut or not changing, even though you're increasing spending on AI. The trend has even been confirmed by one of the most

widely respected and pioneering AI researchers, Ilya Sutskever, who co-founded OpenAI and raised a $1 billion seed round for his new AI startup. As you scale up pre-training,

a lot of the low-hanging fruit was plucked. And so it makes sense to me that you're seeing a deceleration in the rate of improvement, but not everyone agrees the rate of progress has peaked. Foundation model pre-training scaling is intact and continuing.

You know, as you know, this is an empirical law, not a fundamental physical law. But the evidence is that it continues to scale. Nothing I've seen in the field is out of

character with what I've seen over the last ten years or leads me to expect that things will slow down. There's no evidence that the scaling laws, as they're called, have begun to stop. They will eventually stop.

But we're not there yet. And even Sam Altman posting simply, "There is no wall." OpenAI and Anthropic did not respond to requests for comment. Google says it's pleased with its progress on Gemini and has

We've seen meaningful performance gains in capabilities like reasoning and coding. Let's get to the why. If progress is, in fact, plateauing, it has to do with scaling laws; the idea that adding more compute power and more data guarantees better models

to an infinite degree. In recent years, Silicon Valley has treated this as religion. One of the properties of machine learning, of course, is that the larger the brain, the more data we can teach it, the smarter it becomes.

We call it the scaling law. There's every evidence that as we scale up the size of the models, the amount of training data, the effectiveness, the quality, the performance of the intelligence improves. In other words, all you need to do is buy more Nvidia GPUs,

find more articles or YouTube videos or research papers to feed the models, and it's guaranteed to get smarter. But recent developments suggest that may be more theory than law. People call them scaling laws; that's a misnomer. Like Moore's Law, it's a misnomer.

Moore's Law, scaling laws—they're not laws of the universe, they're empirical regularities. I am going to bet in favor of them continuing, but I'm not certain of that. The hitch may be data.

It's a key component of that scaling equation, but there's only so much of it in the world. And experts have long speculated that companies would eventually hit what is called the data wall—that is, run out of it.

If we do nothing, and if you know, at scale, we don't continue innovating, we're likely to face similar bottlenecks in data like the ones we see in computational capability and chip production, or power, and

data center build-outs. So AI companies have been turning to so-called synthetic data; data created by AI, fed back into AI, but that could create its own problem. AI is an industry which is garbage in, garbage out.

So if you feed into these models a lot of AI gobbledygook, then the models are just going to spit out more AI gobbledygook. The information reports that Orion was trained in part on AI-generated data produced by other OpenAI models, and that

Google has found duplicates of some data in the sets used to train Gemini. The problem? Low-quality data, low-quality performance. This is what a lot of the research that's focused on synthetic data is focused on. Right. So if you don't do this well, you

don't get much more than you started with. But even if the rate of progress for large language models is plateauing, some argue that the next phase—post-training or inference—will require just as much compute power. Databricks CEO Ali Ghodsi says there's plenty to build

on top of the existing models. I think lots and lots of innovation is still left on the AI side. Maybe those who expected all of the ROI to happen in 2023, 2024, maybe they, you know, they should

readjust their horizons. The place where the industry is squeezing to get progress is shifted from pre-training, which is, you know, lots of internet data, maybe using synthetic data on huge clusters of GPUs, towards post-training and testing

compute, which is more about, you know, small amounts of data but is very high quality and very specific. Feeding data, testing different types of data, adding more compute. That all happens during the pre-training phase when models

are still being built before it's released to the world. So now companies are trying to improve models in the post-training phase. That means making adjustments and tweaks to how it generates responses to try and boost its performance. And it also means a whole new crop of AI models designed to

be smarter in this post-training phase. OpenAI just announced an improved version of their AI model. They say it has better reasoning. This has been reportedly called 'Strawberry', so there's been a lot of buzz around it.

They're called reasoning models, able to think before they answer. And the newest leg in the AI race. We know that thinking is oftentimes more than just one shot, and thinking requires us to maybe develop multiple plans, multiple potential answers from which we choose the best one

from. Just like when we're thinking, we might reflect on the answer before we deliver the answer. Reflection; we might take a problem and break it down into a step-by-step chain of thought. If AI acceleration is tapped out, what's next?

The search for use cases becomes urgent. Just in the last few weeks, there's been a lot of debate. Have we hit a wall with scaling laws? It's actually good to have some skepticism, some debate,

because that, I think, will motivate more innovation. We've barely scratched the surface of what existing models can do. The models are so powerful today, and we haven't really utilized them to anywhere near their full potential

to bring about true business transformation. OpenAI, Anthropic, and Google are making some of the most compelling use cases yet. OpenAI is getting into the search business. Anthropic is unveiling a new AI tool that can analyze your

computer screen and act on your behalf. One of my favorite applications is NotebookLM. It's a Google application that I used extensively—it's just fun.

But the next phase, the development and deployment of AI agents, is expected to be another game changer for users. I think we're going to live in a world with hundreds of millions, or even billions, of different AI

agents—eventually, probably more AI agents than people. I spoke with Nvidia after the call. They said, 'Jim, you better start thinking about how to use the term "agentic"'

Because "agentic" is the term. Benioff's been using it for a while. He's very agent-focused. You can have health agents, banking agents, product agents, ops agents, sales agents, support agents, marketing agents, and customer experience agents.

and analytics agents, finance agents, and HR agents. And it's all built on this Salesforce platform, meaning it's all powered by software. Everybody's talking about when AI is going to kick in for software? It's happening now.

Well, it has to be. It's not a future thing. It's now; it's something the stock market is already taking note of. Software stocks are seeing their biggest outperformance versus semiconductors in years. And it's key for Nvidia,

which has become the most valuable company in the world and has powered broader market gains. It's hard for me to imagine that Nvidia can grow as fast as people are modeling, and I see that probably as a problem at some point when you get into next year and Nvidia

starts shipping Blackwell in volume, which is their latest chip, and then the vendors can say, okay, we're getting what we need, and now we just need to digest all this money that we've spent because it's not scaling as fast as we thought.

In terms of the improvements, the sustainability of the AI trade hinges on this debate. OpenAI, xAI, Meta, Anthropic, and Google—they're all set to release new models over the next 18 months. Their rate of progress, or lack of it, could redefine the

Are AI advancements already slowing down?

Welcome to the official YouTube channel for CNBC. From 'Wall Street' to 'Main Street' to award winning original documentaries and Reality TV series, CNBC has you covered. Experience special sneak peeks of your favorite shows, exclusive video and more.

For real-time news updates from around the world, visit: http://www.cnbc.com

For CNBC's primetime TV lineup, visit http://tv.cnbc.com

For CNBC's International YouTube channel, visit: http://www.youtube.com/cnbcinternational

CNBC

최근 구글(제미니)과 앤트로픽(클로드 3)의 AI 혁신이 언급되었습니다.

ChatGPT의 미래와 AI 모델 개선 속도가 느려지고 있는 것은 아닌지에 대한 의문이 생깁니다.

주요 기술 기업들의 AI에 대한 막대한 투자에 대해 투자 수익률에 의문을 제기하며 우려의 목소리가 높아지고 있습니다.

실리콘밸리에서는 AI의 빠른 발전이 동력을 잃고 있다는 우려가 커지고 있습니다.

AI의 발전이 규모 측면에서 점근점에 도달하고 있는지에 대한 의문이 제기되고 있습니다.

주요 AI 기업의 발전이 둔화되고 있다는 증거

수십억 달러가 투자되었지만 초기 어려움의 징후가 나타나고 있습니다.

특히 OpenAI의 GPT 모델과 관련하여 최신 AI 모델 간에 큰 진전이 없는 것으로 나타났습니다.

당초 획기적일 것으로 기대되었던 OpenAI의 오리온 모델은 예상보다 작은 개선 효과를 보인 것으로 알려졌습니다.

앤트로픽의 오퍼스 모델도 이전 버전에 비해 성능이 크게 개선되지 않는 등 난관에 부딪힐 조짐을 보이고 있습니다.

구글의 제미니 모델조차 내부 기대에 미치지 못하는 것으로 알려졌습니다.

OpenAI의 공동 창립자인 일리야 수츠케버는 사전 교육에서 아직 열매가 맺히지 않았다고 말합니다.

다른 사람들은 스케일링 법이 여전히 유효하며 진전이 계속될 것이라고 주장합니다.

이 논쟁은 더 많은 컴퓨팅 성능과 데이터를 추가하면 더 나은 모델을 만들 수 있다는 확장 법칙과 이것이 여전히 유효한지에 대한 논쟁을 중심으로 이루어집니다.

'데이터 월'이라는 개념이 언급되며 사용 가능한 데이터의 잠재적 한계에 대해 논의합니다.

합성 데이터의 사용은 잠재적인 해결책으로 논의되고 있지만, AI의 '쓰레기 유입, 쓰레기 배출' 특성으로 인해 잠재적인 문제의 원인으로 지목되기도 합니다.

사후 교육 및 추론으로 초점 전환

일부에서는 AI 개발의 다음 단계는 상당한 컴퓨팅 성능을 필요로 하는 사후 학습과 추론에 초점을 맞출 것이라고 주장합니다.

업계는 사전 교육에서 사후 교육 및 컴퓨팅 테스트에 초점을 맞추고 있습니다.

기업들은 이제 교육 후 단계에서 조정과 수정을 통해 모델을 개선하기 위해 노력하고 있습니다.

OpenAI의 'Strawberry' 모델과 같이 학습 후 단계에서 더 똑똑해지도록 설계된 새로운 AI 모델이 등장하고 있습니다.

AI 사용 사례 및 향후 개발에 집중

기존 AI 모델의 강력한 사용 사례를 찾는 것이 더욱 시급해지고 있습니다.

OpenAI의 검색 진출과 Anthropic의 새로운 AI 도구 등 OpenAI, Anthropic, Google의 사용 사례를 예로 들어 설명합니다.

AI 에이전트의 개발과 배포는 업계의 판도를 바꿀 것으로 예상됩니다.

AI가 소프트웨어에 미치는 영향에 대해 논의되고 있으며, 주식 시장에서는 이미 주목하고 있습니다.

AI 개선의 확장 속도가 예상보다 느릴 수 있다는 점을 고려할 때 Nvidia의 미래 성장에 의문이 제기되고 있습니다.

Are AI advancements already slowing down?

Are AI advancements already slowing down?

스크립트