Generative AI's Impact on Data Centers

The demand for powerful servers in data centers is soaring due to the insatiable appetite for computing in the cloud, driven by AI applications.

### The Growing Power and Water Demands of AI Data Centers
* 🌐  Data centers are experiencing unprecedented demand due to the rise of generative AI models like ChatGPT, with one query consuming **10 times more energy than a Google search** and as much as **keeping a 5-watt LED bulb on for an hour**.
* 🏭  The AI frenzy is driving data center demand to rise **15-20% annually through 2030**, potentially reaching **16% of total US power consumption by 2030**.
* 💧  AI's water consumption is also a concern, with global AI demand projected to **withdraw up to 6.6 billion cubic meters of water annually by 2027**.
* 🔌  The power needs of AI data centers are so high that some plans to **close coal-fired power plants** are being put on hold, and the aging grid is **struggling to keep up**.
* 💡  Solutions being explored include building data centers in areas with abundant power, generating power on-site through **solar, nuclear, or other means**, and improving chip efficiency to **reduce compute needs**.
* 🏢  ARM-based processors are gaining popularity in data centers due to their **power efficiency**, with companies like AWS reporting **60% power savings** compared to competitive architectures.
* 🌊  Some data centers are using innovative cooling solutions to reduce water consumption, such as **air-based cooling systems** and **direct liquid cooling for chips**.
* 🌱  Despite these challenges, the industry is optimistic about finding solutions to meet the growing demand for AI computing, with ongoing research and development focused on **improving efficiency and sustainability**.

Demand has never been higher for these racks and racks
of powerful servers, feeding the internet's insatiable appetite for computing in the cloud. The reality is that the cloud is not up there
somewhere. It's right here. We are in it. You're in the middle of the cloud as we speak. And data centers like this can't ever stop streaming
social media, photo storage, and more recently, and requiring much more data training and running
chatbots like OpenAI's ChatGPT, Google's Gemini, and Microsoft's Copilot. And you can feel the heat coming off of these. Thanks to the generative AI race, data centers like
this are springing up as quickly as companies like Vantage can build them, and that means demand for
power to run them and cool them is through the roof, too. If we don't start thinking about this power problem
differently now, we're never going to see this dream we have or the potential we have of this
amazing technology that can truly

change our lives. One ChatGPT query takes nearly ten times as much energy
as a typical Google search, and as much energy as keeping a five-watt LED bulb on for an hour. Generating an AI image can use as much power as
charging your smartphone. Hyperscalers building data centers to accommodate AI
have seen emissions skyrocket, and this problem isn't new. Estimates way back in 2019 found training one
large language model produced as much CO2 as the entire lifetime of five gas-powered cars. And even if we can generate enough power, our aging
grid is increasingly unable to handle the load. If you look at the peak demand during the summer time,
if the data centers don't reduce their load, there could be a blackout. With the looming question of whether we'll have enough
power for the widespread adoption of generative AI, Cnbc visited a data center in Silicon Valley to see
those massive compute loads firsthand, and talk to

those at the center of the problem to find out what
can be done. There are more than 8000 data centers globally, with
the highest concentration in the US. But it's not enough. But we suspect that the amount of demand that we'll see
from AI-specific applications will be as much or more than we've seen historically from cloud
computing. The AI frenzy has data center demand rising 15 to 20%
every year through 2030. And as companies like Vantage build more and more,
getting enough power is key. Data centers could reach a whopping 16% of total US
power consumption by 2030, according to one report, up from just 2.5% before ChatGPT hit the scene
in 2022. That's the equivalent of about two-thirds of the total
homes in the US. Natural gas is expected to fuel the majority of this,
and utilities will need to invest some 50 billion to support the growth.

Data centers that probably around 64MW for the building itself. Many of those are being taken up by single customers. They'll have the entirety of the space leased to them. And as we think about AI applications, those numbers can grow quite significantly beyond that into hundreds of megawatts. 64mw, 100MW. How many homes, you know, on average. Tens and tens of thousands of homes per data centers worth of power. Many big tech companies contract with companies like Vantage to house their servers, but the needs of some have grown so much that many have been building their own data centers. For Google and Microsoft, this has directly translated to soaring emissions. Google's latest environmental report says greenhouse gas emissions rose nearly 50% from 2019 to 2023, in part because of data center energy consumption, although it adds its data centers are 1.8 times as energy efficient as a typical data center.

Microsoft's emissions rose nearly 30% from 2020 to 2024, due to data centers designed and optimized to support AI workloads. Power needs are so high that some plans to close coal-fired power plants are being put on hold, like in Kansas City, where Meta is building an AI-focused data center. So you can never really stop the AI computing from growing. So they will do whatever they can to get the power capacity. One approach: building data centers where power is more plentiful. The industry itself is looking for places where there is either proximate access to renewables, whether wind or solar, and other infrastructure that can be leveraged, whether it be part of an incentive program to convert what would have been a coal-fired plant into natural gas, or increasingly looking at ways in which to offtake power from nuclear facilities. Santa Clara, where we visited Vantage, has long been one of the nation's hot spots for clusters of data

centers near data-hungry clients. Nvidia's headquarters was visible from the roof. Now, Vantage is building in places like Columbus,
Ohio, and Atlanta, Georgia. In Northern California, we're seeing a bit of a
slowdown in terms of where data centers are deployed because of the lack of
availability of power from the utilities here in this area. On the flip side of bringing data centers where the
power is, some AI companies and data centers are experimenting with ways to generate their own power
right on site. OpenAI CEO Sam Altman has been vocal about this need. He recently invested in a solar startup that makes
shipping container-size modules that include both the panels and power storage in one. Altman's also invested in nuclear fission startup Oklo
that aims to make mini nuclear reactors housed in a frame structures and in nuclear fusion startup
Helion. Microsoft also signed a deal with Helion last year to
start buying its fusion electricity in 2028.

and Google's partnered with a geothermal startup that
says its next plant will harness enough power from deep underground to run a large data center. Even data centers themselves are starting to generate
their own power. Which Vantage has done, for example, in Virginia, where
over the course of the last year, we deployed 100-megawatt natural gas power plant to support a
dedicated data center for one of our customers. There, we're self-contained, and we're delivering power
to the data center that we have there, and it doesn't touch the public grid at all. Even when enough power can be generated, the aging grid
is often ill-equipped to handle transmitting it. And that's where grid hardening comes in. With concentrations of data centers in particular
areas, there tends to put somewhat more pressure on the grid in terms of delivering that
much power in those locations. In an area of northern Virginia known as data center
alley, servers process an estimated 70% of the world's

internet traffic each day. At one point in 2022, the power company there had to
pause new data center connections as it struggled to keep up with demand. Basically, during the peak hour, we either ask the residents to turn off their AC, or we ask the AI company to stop their training. Vantage, as an example, voluntarily supports a load shedding program such that when the utility knows that they're going to have
a constraint based on high temperatures, people need to run their air
conditioners. We voluntarily come off the grid. We run our own generators during that time in order to
ensure that that power is available for everybody. The bottleneck often occurs in getting power from the
site where it's generated to where it's consumed. One solution is to add hundreds or even thousands of
miles of new transmission lines. But those projects, like 15. $2 billion effort to expand lines to Data Center
Alley, have been met with opposition from local

ratepayers who don't want to see their bills go up to
fund the project. It is a part of the solution to increase the capacity
from the grid side, but that's very costly and very time-consuming, and sometimes the cost is just passed
down to residents in there in term of their utility bill increase. Another solution is to use predictive software to
reduce failures at one of the grid's weakest points the transformer. All electricity generated must go through a
transformer, and that's primarily its function connects two sides of electrical circuits together. There are 60 to 80 million transformers in the US
alone. The average transformer in the US is 38 years old, so
the aging devices have become a common cause for power outages, and replacing them is expensive and
slow. So we have this tiny little sensor. It's two-inch diameter, one-inch thickness, about half
the size of a hockey puck, and we essentially glue it on the outside of a transformer.

Software can then predict failures and determine which
transformer can handle more load, so it can be shifted away from those at risk of failure. It's been installed in high-power demand areas like
part of Mongolia, where data centers are coming online. Vi says business has tripled since ChatGPT came on the
scene in 2022. And what we're hearing in 2024 is that we might double or
triple again next year. A large reason we need more power and a more reliable
grid for generative AI is to keep its servers cool. All the servers generate an immense amount of hot air,
and cooling them down with air or water keeps them from overheating so they can keep running 24/7. The problem is, AI is projected to withdraw more water
annually by 2027 than four times all of Denmark. Everybody is worried about AI being energy-intensive. We can solve that when we get off our ass and stop
being such idiots about nuclear.

Right. That's solvable. Water is the fundamental limiting factor to what is coming in terms of AI. I was quite shocked when I saw the number of AI computing's water needs. Shaolei Ren's been studying data center efficiency for over a decade. His research team found that every 10 to 50 ChatGPT prompts can burn through what you'd find in a standard 16-ounce water bottle. Training takes even more power and generates even more heat as it learns by accessing all the data constantly being added to the entire internet. Training GPT three in Microsoft's US data centers can directly evaporate 700,000L of clean, fresh water. With global AI demand accountable for up to 6,600,000,000m³ of water withdrawal by 2027, more than four times the total annual withdrawal of all of Denmark. In drought-ridden Chile, the government partially reversed Google's permit to build a data center there following public outcry about water.

usage, and it's facing similar backlash to planned
data centers in Uruguay, too. There are technologies that are used in certain parts
of the industry that do consume water in order to cool data centers through evaporative cooling. That tends to be very efficient from a power
perspective; it tends to be very inefficient from a water use perspective. And so vantage is designed from the beginning, really,
has been to avoid using water at all costs. Instead, Vantage uses dozens of gigantic air
conditioning units on the roof. There's a combination of air handling and cooling
towers at the top there, where the condenser coils allow heat that's coming up from the hot aisles that
we were looking at inside the data hall, and then to be converted back into the chilled water loop and brought
back down into those critical air handlers. Microsoft has halted a radical project that tried
keeping its servers cool by submerging them under the ocean. Another solution is using cool liquid directly
on chips to cool them rather than the

inefficient process of cooling air with water. People are working on a whole bunch of things like
director chip cooling, for example, which would radically reduce the amount of water that's needed. But the speed at which this is building makes it very
challenging indeed. So there's a lot of talk in the industry right now in
support of AI. Moving. From air, which is moving around us to water, air or
liquid that can be brought to the chip itself in order to cool it in a much more
efficient way. But for a lot of data centers, that requires an
enormous amount of retrofit in our case, advantage. About six years ago, we deployed a design that would
allow for us to tap into that cold water loop here on the data hall floor. There is one broader approach to solving. AI's massive problem of water and power lessen the
amount of compute it needs. In other words, get more work per watt.

It all comes back to how do I get more done with the
power supply I have? Scale flux makes memory and storage devices for data
centers focused on power savings. Data compression can be very slow, latency adding, and
power-hungry if you try to do it. In general-purpose cores like Intel or AMD x86
processors, when you put it into a hardware state machine, it can
be done orders of magnitude faster and more efficiently, more power-efficient. The key alternative to those power-hungry x86 cores are
ARM-based specialized processors. Arm got its start making low-powered chips that
maximized the battery life of early mobile phones, saving every last bit of power. It's going to be a fundamentally different design than
when you're trying to maximize the performance, and so it could be something as simple as how we
access our memory. It can be as simple as how we access data, but every
moment of that architecture is thought of as power first.

Now ARM makes all sorts of chips, including Neoverse for data centers. And as AI takes off, ARM's power efficiency has made it increasingly popular with tech giants like Google, Microsoft, Oracle, and Amazon. I have here AWS Graviton, and AWS Graviton saves 60% power versus competitive architectures. Nvidia's latest AI chip unveiled in March, Grace Blackwell, uses ARM-based CPUs that it says can run gen AI models on 25 times less power. And ARM says a data center filled with its chips can use 15% less power because of lower compute needs. If you think about the scale of data centers, 15% is almost 2 billion ChatGPT queries. Think about that. You can light 20% of American households with just that 15% savings. And companies like Apple, Samsung, and Qualcomm have been touting the benefits of doing AI on device. A huge energy savings for each query kept off the cloud off of servers like these, perhaps giving data

centers need time to build more and catch up to AI's insatiable appetite for power. Everybody will build the data centers that they can, and we'll have as much AI as those data centers will support, and it may be less than what people aspire to. But ultimately, there's a lot of people working on finding ways to throttle some of those supply constraints, but definitely a lot of growth ahead in this industry.

How The Massive Power Draw Of Generative AI Is Overtaxing Our Grid

Welcome to the official YouTube channel for CNBC. From 'Wall Street' to 'Main Street' to award winning original documentaries and Reality TV series, CNBC has you covered. Experience special sneak peeks of your favorite shows, exclusive video and more.

For real-time news updates from around the world, visit: http://www.cnbc.com

For CNBC's primetime TV lineup, visit http://tv.cnbc.com

For CNBC's International YouTube channel, visit: http://www.youtube.com/cnbcinternational

CNBC

The Growing Power Demand of AI Data Centers

The demand for data centers to support AI applications, like generative AI and language model training, is rapidly increasing, leading to a significant surge in power consumption.

This escalating power demand is projected to potentially reach 16% of the total US power consumption by 2030, mainly fueled by natural gas.

The rapid expansion of data centers, driven by the AI frenzy, is raising concerns about the ability of the aging grid to handle the load, with the potential risk of blackouts if the demand is not addressed.

The Impact of AI on Data Center Power Consumption

Tech companies' growing AI applications are leading to a significant increase in data center power needs, causing soaring emissions and impacting the environment.

This has resulted in a shift towards building data centers in areas with access to plentiful power, including a focus on renewable energy sources and even the exploration of generating their own power on-site.

Additionally, investments in nuclear energy startups and a future deal for fusion electricity signify a potential transition towards more sustainable power solutions for data centers.

Google's Partnership with Geothermal Startup for Data Center Power

Google is collaborating with a geothermal startup to power a data center using energy from deep underground, while some data centers are now generating their own power.

The concentration of data centers, particularly in areas like northern Virginia, is putting a strain on the power grid, leading to opposition to the expansion of transmission lines.

To address the grid's weaknesses, predictive software is being considered to reduce failures in the aging transformers that are a common cause of power outages.

The Impact of AI on Water Usage and Cooling Technologies

AI, particularly ChatGPT, is projected to have a significant impact on global water withdrawal, with its computing processes potentially using up to 6,600,000,000m³ of water by 2027.

The demand for water-intensive data centers is facing backlash in drought-affected areas like Chile and Uruguay.

To address the issue, some companies are shifting towards water-free cooling technologies, such as using air conditioning units instead of water-based systems.

Innovations in Air Cooling for Data Centers

Efforts are underway to improve the efficiency of air cooling in data centers, including the development of direct chip cooling and the use of AI to shift from air to liquid cooling.

Retrofitting existing data centers for these cooling methods poses a significant challenge due to the required infrastructure changes.

Another approach to addressing the water and power consumption of data centers is to reduce the amount of compute needed, focusing on getting more work done per watt.

This can be achieved through the use of specialized, power-efficient processors like ARM-based chips, which prioritize power savings in their design.

ARM's Growing Influence in Data Centers and AI

ARM's power-efficient chips, like Neoverse, are increasingly popular among tech giants for data centers and AI, with AWS Graviton saving 60% power.

Nvidia's latest ARM-based AI chip, Grace Blackwell, boasts significant power savings, and ARM claims its chips can reduce power usage in data centers by 15%.

The use of ARM chips for AI on devices is also promoted by companies like Apple, Samsung, and Qualcomm, offering significant energy savings and potentially allowing data centers to catch up to AI's power demands.