Google의 TPU 개발: 혁신과 산업에 미치는 영향

### Google의 맞춤형 칩 AI 프로세싱의 진화와 미래
* 🖥 구글의 실리콘밸리 본사에는 **검색과 YouTube 광고를 비롯한 다양한 구글 서비스를 구동하는 데 중요한 최신 텐서 처리 장치(TPU)를 테스트하는 연구소**가 있습니다.
* 🔍 **256개의 칩이 탑재된 최초의 트릴리움 시스템**은 거의 10년 전부터 음성 인식 기능을 향상시키기 위해 시작된 맞춤형 칩 개발에 대한 Google의 노력을 잘 보여줍니다.
* 💡 구글은 원래 '텐서 프로세싱 유닛'이라는 용어를 만들었으며, 이후 이 분야에서 **58%의 시장 점유율**을 차지하며 맞춤형 AI 칩의 선두주자가 된 반면, 아마존과 마이크로소프트 같은 경쟁업체는 최근에야 이 분야에 뛰어들었습니다.
* ⚡ AI 하드웨어 분야에서 선구적인 역할을 해왔음에도 불구하고, 구글은 특히 **훈련에 TPU를 활용하는 챗봇 제미니의 출시가 지연**되는 등 제너레이티브 AI 경쟁에서 뒤처지고 있다는 비판에 직면해 있습니다.
* 💧 또한 데이터 센터의 에너지 효율을 개선하고 물 사용량을 줄이기 위해 상당한 조치를 취했으며, 환경 문제를 해결하기 위해 **직접 칩 냉각 시스템**을 구현했습니다.
* 🌍 AI 기능에 대한 수요가 증가함에 따라 Google은 경쟁사에 비해 시장 진입이 늦었지만 곧 출시될 **범용 CPU인 Axion**을 비롯한 칩 개발에 막대한 투자를 하고 있습니다.
* 🔗 특히 지정학적 위험을 고려할 때 칩 설계 및 제조의 복잡한 특성을 강조하는 **복잡한 AI 칩**을 만드는 데 있어 **Broadcom과 같은 회사와의 파트너십**은 Google에게 필수적이었습니다.
* 🚀 AI 산업이 빠르게 발전하는 가운데, Google은 **AI 애플리케이션의 증가하는 연산 수요**를 충족하기 위해 칩 기술을 발전시키는 데 주력하고 있으며, 이는 **맞춤형 실리콘 이니셔티브의 유망한 미래**를 시사합니다.

Google의 맞춤형 AI 칩, 특히 TPU는 클라우드 컴퓨팅에 혁신을 가져왔으며 AI 환경에서 경쟁 우위를 점하고 있습니다.

At this sprawling lab at Google's Silicon Valley headquarters, these racks and racks of servers aren't running workloads for Google Cloud's millions of customers. Here, for example, is the very first Trillium system that we built is a full Trillium system with 256 chips in it for racks. Or for YouTube, or the world's most dominant search engine. And what is Trillium? Trillium is our latest generation TPU. It'll be public later this year. Instead, they're running tests on its very own microchips, Tensor Processing Units, that help power it all, search. And of course, video, YouTube ads. Everything Google does has been powered in many ways by its own homegrown TPU. Now, TPUs are used to train AI models like Google's own chatbot Gemini. .. and in some big news, Apple's AI, too. Apple, actually, we found out yesterday they disclosed in a paper they're using Google-made chips.

The world sort of has this fundamental belief that all AI large language models are being trained on Nvidia. But Google took its own path here. And yet, despite being the birthplace of some foundational concepts behind generative AI, many say Google's fallen behind in the AI race. But it was the first major cloud provider to do custom AI chips. It was ten years ago, almost to the day, where we decided that to meet the needs of our users in terms of a particular application, voice recognition at the time, we needed to design custom hardware. In the years since, Amazon, Microsoft, and Meta have started making their own AI chips too. Here we're turning on the chips and the boards for the first time, making sure they're working properly to specification, debugging any issues that might come up, that sort of thing. And no media has been inside here before. First time? Yep. We went to Google headquarters for an exclusive look inside the chip lab and sat down with its top executive.

to ask why and how. Google's betting big on the expensive, complex business of custom chips. It all started in 2014 when a group at Google calculated that in order to launch upcoming voice recognition features, Google would need to double the number of computers in its data centers. Amin Vahdat, now the head of custom cloud chips, started at Google four years before that. A number of leads at the company asked the question, what would happen if Google users wanted to interact with Google via voice for just 30s a day? And how much compute power would we need to support our users? We realized that we could build custom hardware, not general-purpose hardware, but custom hardware Tensor Processing Units in this case to support that much, much more efficiently. In fact, a factor of 100 more efficiently than it would have been otherwise. What is a Tensor Processing Unit? And did you guys coin that term? We did. We coined the term Tensor Processing Unit.

We believe that it was certainly the first large-scale hardware accelerator for AI applications. There's a whole gamut of qualification and validation tests we do on, you know, power thermals functionality. You're really trying to make sure the design has enough margin so that it's going to operate well, you know, at volume, at scale. Principal engineer Andy Swing, who ended up leaving Google since our visit, was there for the first launch. There's actually four chips inside there. It's connected to. Actually, two of those are connected to a host machine that has CPUs in it. And then all these colorful cables are actually linking together all of the Trillium chips to work as one large supercomputer. Google data centers still rely heavily on chip giants like Intel and AMD for Central Processing Units (CPUs) and Nvidia for Graphics Processing Units (GPUs). Google makes a different category of chips called ASICs (application-specific integrated circuits), which are more efficient because they're built for a single purpose.

Google's best known for its AI-focused ASIC, the TPU. But it also makes ASICs to power YouTube, called VCUs, Video Coding Units. And just like Apple, Google also makes custom chips for its devices. The G4 powers the new, fully AI-enabled Pixel 9, and the new A1 powers Pixel Buds Pro 2. But the TPU is what set Google apart because when it launched in 2015, it was the first of its kind. So the AI cloud era has completely reordered the way companies are seen. And this silicon differentiation, the TPU itself may be one of the biggest reasons that Google went from the third cloud to being seen truly on parity, and in some eyes, maybe even ahead of the other two clouds for its AI prowess. Amazon Web Services announced its first cloud AI chip, Inferentia, in 2018, three years after Google's came out. Microsoft's first custom AI chip, Maya, wasn't announced until the end of 2023.

In order to stay differentiated, to stay competitive, to stay ahead of the market, and to not become overly dependent on any supply chain, partner, or provider, they needed to do more, build more in-house. According to Newman's team's research, Google TPUs dominate among custom cloud AI chips, with 58% of the market share, and Amazon comes in second at 21%. In 2017, a group of eight Google researchers wrote the now-famous paper that invented the transformer, the underpinnings of today's generative AI craze. The invention, Vahdat says, was made possible by TPUs. The transformer computation is expensive, and if we were living in a world where it had to run on general-purpose compute, maybe we wouldn't have imagined it. Maybe no one would have imagined it. But it was really the availability of TPUs that allowed us to think, not only could we design algorithms like this, but we could run them efficiently at scale.

Still, Google has faced criticism for some botched product releases in the current rat race of generative AI, and its chatbot, Gemini, came out more than a year after OpenAI's ChatGPT. Dozens and dozens of customers are leveraging Gemini every day, including some of the most familiar names out there, whether it's Deutsche Bank, Estée Lauder, and many, many others that are household names, McDonald's, if you like, and others. Was Gemini trained on TPUs? Gemini was trained and is served externally entirely on TPUs. Back in 2018, Google expanded the focus of TPUs from inference to training AI models. Version two was actually a pod that connected 256 TPUs together. Now version five is in production, which connects almost 9000 chips together. The real magic of this TPU system is that you actually can interconnect everything over fiber optics dynamically. And so you can build small or as large of a system as you want.

With version two in 2018, Google also made its TPUs available to third parties alongside market-leading chips like Nvidia's GPUs, which are still used by most cloud customers. If you're using GPUs, they're more programmable, they're more flexible, but they've been in tight supply. The AI boom has sent Nvidia's stock through the roof, catapulting the chipmaker to a $3 trillion market cap in June, surpassing Google's parent company Alphabet, and jockeying with Apple and Microsoft for position as the world's most valuable public company. Being candid, these specialty AI accelerators aren't nearly as flexible or as powerful as Nvidia's platform, and that is what the market is also waiting to see is can anyone play in that space? Now that we know Apple's using Google's TPUs to train its AI, the real test will come as it rolls out those full AI capabilities on iPhones and Macs next year.

They were renting chips from Google for about two bucks an hour, times a gazillion chips to train their AI models. So they didn't even need Nvidia. All the market pull is coming from Nvidia, but longer-term, people are just going to want to do AI things. And when they want to just do AI things, they may be just as happy to do it on a TPU or do it on another homegrown piece of AI dedicated silicon. But developing alternatives to Nvidia's hugely powerful and expensive chips is no small feat. It's expensive. You need a lot of scale. And so it's not something that everybody can do. But these hyperscalers, they've got the scale and the money and the resources to go down that path. But the process is so complex and costly that even the Googles of the world can't do it alone. Since the very first TPU ten years ago, Google's partnered with Broadcom, a chip developer that also

helps Meta design its AI chips. Broadcom says it's spent more than $3 billion on R&D to make these partnerships happen. AI chips, they're very complex. There's lots of things on there. So Google brings the compute. Broadcom does all the peripheral stuff. They do like the I/O and the SerDes and all of the different pieces that go around that compute. They also do the packaging. Then the final design is sent off to be manufactured at a fabrication plant or fab, primarily those owned by the world's largest chip maker, Taiwan Semiconductor Manufacturing Company, which makes some 92% of the world's most advanced semiconductors. Do you have any safeguards in place should the worst happen in the geopolitical sphere between China and Taiwan? Yeah, it's an important question. And it's certainly something that we prepare for and we think about as well. But we're hopeful that actually it's not something that we're going to have to trigger.

I think the entire world is at the same risk. It's not unique to Google. It's not unique to Amazon. It's not unique to Apple. It's not unique to Nvidia. If Taiwan is not given the appropriate support, if it deals with unexpected end-of-day circumstances, it is not only going to set back any one of these companies; it's going to set back the whole world. That's why the White House is handing out $52 billion in CHIPS Act funding to companies building fabs in the US, with the biggest portions going to Intel, TSMC, and Samsung, so far. Intel and TSMC are putting a lot of their own money into this as well. I'm heartened to see that. But I mean, it's going to take a long time to duplicate. So let's hope that it doesn't need to be duplicated. Risk aside, Google just made another big chip move, announcing its first general-purpose CPU.

Axion will be available by the end of the year. Now we're able to bring in that last piece of the puzzle, the CPU. And so a lot of our internal services, whether it's BigQuery, whether it's Spanner, YouTube, advertising, and more, are running on Axion. But Google is late to the CPU game. Amazon launched its processor Graviton in 2018. Alibaba launched its own server chip in 2021, and Microsoft announced its CPU in November. Why didn't you do it sooner? Our focus has been on where we can deliver the most value for our customers, and there it has been, starting with the TPU, our video coding units, our networking. We really thought that the time was now, starting a couple of years ago. Again, these things are a number of years in the making to really bring our expertise to bear on the ARM CPUs. I don't fault Google for pacing out the launch of Axion in a more

delayed fashion. This wasn't as differentiated. It's not as differentiated. To me, it is more of a supply game. It's more of a margin and vertical integration game for the company. Whereas the TPU was truly differentiated. Six generations, ten years of experience. All these processors from non-chip makers, including Google's, are made possible by Arm Chip architecture, a more customizable, power-efficient alternative that's been gaining traction over the traditional x86 model from Intel and AMD. Power efficiency is crucial because, by 2027, AI servers are projected to use up as much power every year as a small country. With TPUs, the ability to customize greatly boosts power efficiency. This is our second-generation optical circuit switch. So, our large TPU supercomputers are actually optically interconnected. It allows us to dynamically link together collections of TPU chips to custom-tailor the dimensions to the job that's running. This is developed all in-house by us.

Power is a huge thing now, and you know anything you can do to try to improve efficiency, lower costs, and save power, I think you're going to do. Google's latest environmental report showed emissions rose nearly 50% from 2019 to 2023, partly due to data center growth for powering AI. Without having the efficiency of these chips, the numbers could have wound up in a very different place. We remain committed to actually driving these numbers in terms of carbon emissions from our infrastructure 24/7, driving it towards zero. Training and running AI also takes a massive amount of water to keep the servers cool so they can run 24/7. That's why with the third generation of TPU, Google started using direct to chip cooling, a new way to cool servers that uses far less water, and that's also being used by Nvidia's latest Blackwell GPUs. We have four chips, and these are our liquid cooling lines that come in.

There's essentially a cold plate here that has little fins in it, and it picks up the heat from the chip, puts it into the water, and that comes back out. Despite challenges from geopolitics to power and water, Google is committed not only to its generative AI tools but to making its own chips to handle the massive compute required by the craze. I've never seen anything like this, and no sign of it slowing down quite yet. I think it's fair to say that we really can't predict what's going to be coming as an industry in the next five years, and hardware is going to play a really important part there.

How Google Makes Custom Cloud Chips That Power Apple AI And Gemini

Welcome to the official YouTube channel for CNBC. From 'Wall Street' to 'Main Street' to award winning original documentaries and Reality TV series, CNBC has you covered. Experience special sneak peeks of your favorite shows, exclusive video and more.

For real-time news updates from around the world, visit: http://www.cnbc.com

For CNBC's primetime TV lineup, visit http://tv.cnbc.com

For CNBC's International YouTube channel, visit: http://www.youtube.com/cnbcinternational

CNBC

Google의 실리콘밸리 연구소는 검색 및 YouTube와 같은 서비스를 구동하는 자체 텐서 처리 장치(TPU)를 테스트하고 있습니다.

TPU는 Google의 챗봇 제미니와 최근에는 Apple의 AI를 비롯한 AI 모델을 훈련하는 데도 사용됩니다.

구글의 맞춤형 AI 칩 개발은 거의 10년 전부터 시작되었으며, 이후 아마존과 마이크로소프트 같은 다른 거대 기술 기업들도 이를 따르도록 영향을 미쳤습니다.

Google이 AI 하드웨어에 미치는 영향

Google은 대규모 AI용 하드웨어 가속기를 최초로 출시한 회사로, TPU(애플리케이션별 집적 회로)를 통해 클라우드 AI 시장에서 차별화된 경쟁력을 확보했습니다.

TPU의 효율성과 가용성은 오늘날의 제너레이티브 AI를 뒷받침하는 트랜스포머 모델을 발명하는 데 결정적인 역할을 했습니다.

Google은 자체 칩 생산을 통해 58%의 점유율로 맞춤형 클라우드 AI 칩 시장을 장악할 수 있었습니다.

구글의 제미니와 TPU: AI의 게임 체인저

ChatGPT 이후 1년여 만에 출시된 Google의 Gemini 챗봇은 현재 도이치뱅크와 맥도날드 같은 주요 기업에서 사용하고 있습니다.

이 챗봇은 256개에서 거의 9000개에 달하는 칩으로 발전한 Google의 TPU 시스템을 통해 학습되고 제공됩니다.

애플이 AI 트레이닝에 구글의 TPU를 사용하면서 엔비디아의 지배적이고 값비싼 칩을 대체할 수 있는 대안을 모색하는 시장의 관심이 높아지고 있습니다.

Google은 AI 칩 설계를 위해 Broadcom과 협력하고 있으며, Broadcom은 30억 달러 이상을 R&D에 투자하고 있습니다.

복잡한 칩 제조 공정은 전 세계 첨단 반도체의 92%를 생산하는 대만 반도체 제조 회사에 의존하고 있습니다.

구글은 자사 최초의 범용 CPU인 Axion으로 뒤늦게 CPU 시장에 진출하여 내부 서비스를 강화하는 것을 목표로 하고 있지만, 자체 프로세서를 출시하면서 아마존, 알리바바, 마이크로소프트와의 경쟁에 직면하고 있습니다.

Google의 영향력 있는 칩 아키텍처와 AI 전력 효율성

Google의 칩 아키텍처, 특히 Arm 칩은 커스터마이징과 전력 효율성으로 주목받고 있으며, 이는 AI 서버의 수요 증가에 중요한 역할을 합니다.

Google의 TPU 슈퍼컴퓨터는 광 상호 연결을 사용하여 칩을 동적으로 연결함으로써 전력 효율을 높이고 비용을 절감합니다.

탄소 배출량을 줄이고 서버에 직접 칩 냉각을 사용하려는 Google의 노력은 AI에 필요한 대규모 컴퓨팅을 처리하는 동시에 지속 가능성을 위한 헌신을 보여줍니다.