Grok's Strategic Focus: Inference and Growth

Lisa Sue, the AMD CEO, raised substantial funds to build LPG chips, hire, offer token services, and focus on inference for Grok's AI business.

### Groq CEO on the Importance of Inference in AI and Strategic Partnerships
* 💰  Groq plans to use recently raised funds to build 180,000 LPG chips, equivalent to about **20% of Nvidia's Hopper GPU shipments last year**.
* 👩‍💼  CEO Lisa Su emphasizes the importance of **inference**, as training is a well-solved problem, but the cost of putting models into production is **10 to 20 times** that of training.
* 🏗️  The company is expanding its "**token as a service**" business model, allowing users to access hardware without purchasing it, and is **considering acquisitions**.
* 🤝  Groq's focus on inference from the beginning is strategic, as CEO Jonathan Ross explains, "**You spend money to make the models; you make your money with inference**."
* 👨‍💻  The appointment of John McCanny, a Turing Award winner and advocate for **open source AI models**, as a technical advisor is significant for Groq.
* 🌐  Ross highlights the importance of open source models, stating, "**Grok actually wouldn't exist today if it wasn't for open source models**."
* 🚀  Open weights models, similar to Meta's Lamar models, are described as "**the Linux of the generative age**" by Ross, enabling broader access to AI technology.

Just last week, we were speaking to Lisa Sue, the AMD CEO, about the move from training to influence. It's a sizeable chunk of change she managed to raise. I guess the best starting place what we use the cash for. The first thing is that we're going to build out about 180,000 of our LPG chips or language processing units. To put that in perspective. You mentioned Hopper, the Nvidia GPU. Last year they shipped about 500,000, so this would be about 20% of what they shipped last year. Secondly, we intend to hire significantly. We're building out a lot of features. We actually offer a token as a service business where people can just use our hardware without having to buy it. And then we are looking at some acquisitions going forward as well. Strategically, when did Grok sort of say, okay, inference needs to be our focus and how is it helping you get sort

of get ahead?
There's lots been discussed on this program about the many startups and
companies working on training large language models for many hundreds of
billions of parameters to smaller models.
You have in more narrow focus, Jonathan. Yeah.
So Gronk from the beginning is focused on inference, and the reason is training
at this point is a well-solved problem. And we always knew that it would be my
background. I actually started to Google TPU chip,
and we actually started with inference there because we were actually able to
train the models. We just couldn't afford to put them in
production. Typically, whatever you spend on
training, you will continue to spend going forward, but you're going to spend
10 to 20 times that much putting it into production.
You spend money to make the models; you make your money with inference.
It's also a scale game. And so we've been focused on this since
day one. There's also sort of a relationship and
talent conversation to be had.

Yang looking over at Metta is going to be your new technical advisor, which was in the news as well. Why is that important to you? So John McCain is legendary in this industry. He won the Turing Award for being one of the three people who invented the algorithms that make modern A.I. work. And of course, John McCain is very centrally positioned in terms of A.I. He's also pushing very heavily for open source models or open weights models. And this is something that's important for the industry. Grok actually wouldn't exist today if it wasn't for open source models. We'd actually built the world's best chips. But if we didn't also have the software to run on it, we wouldn't be able to demonstrate that. And so the Lamar models for Meta effectively are the Linux of the generative age, and it's enabling people to get into the AI business without having to train the models themselves.

Groq's $2.8 Billion Valuation Tamps Down AI Fears

Welcome to the Bloomberg Technology YouTube channel. Subscribe for original series and mini-docs on game-changing technologies, profiles on innovators, live conversations with experts, and breaking technology news & analysis from Bloomberg TV.

Bloomberg Technology

AMD's Plans for the Future: Chip Production, Hiring, and Acquisitions

AMD CEO Lisa Sue raised a significant amount of money to expand LPG chip production and hire more staff.

The company also plans to offer a token-as-a-service business and pursue acquisitions in the future.

The shift towards influence as a focus has been strategic for AMD's growth in the inference market.

The Focus on Inference in Training Language Models

Gronk's focus on inference is driven by the high cost of putting trained models into production.

The company recognizes the importance of making money through inference rather than just spending on training.

Additionally, Gronk emphasizes the significance of scale and talent in the development of language models.

John McCain, a recipient of the Turing Award, played a crucial role in inventing the algorithms for modern AI and is a central figure in the industry.

He is a strong advocate for open source AI models, which is important for the development of the industry.

His work has been instrumental in enabling more people to enter the AI business by providing access to Meta's Lamar models, which are compared to the Linux of the generative age.