### Energy Efficient AI: Disrupting GPU Technology Landscape

Innovative startups are revolutionizing AI efficiency by significantly reducing power consumption and deployment costs.

### Revolutionizing AI Efficiency: A New Era of Dataflow Architecture
* 🔋 "Power and energy is going to be a **big bottleneck**" as the demand for video production and AI scaling increases, highlighting the need for **efficient computing solutions**.
* 🚀 Originating from Stanford University, the startup focuses on **enhancing energy efficiency in AI applications**, allowing for **significant scaling without hitting power limitations**.
* 🏠 A single rack of NVIDIA technology consumes over **100 kilowatts**, equivalent to powering **100 homes**, whereas the startup's technology can achieve **ten times the performance** at just **one-tenth the power**.
* ⚡ This advancement is made possible through a **novel dataflow architecture** that aligns hardware with neural network operations, **eliminating the energy-intensive process** of breaking up neural networks as required by traditional GPUs.
* 🏢 The technology seamlessly integrates into existing data centers without the need for **costly upgrades**, providing **immediate value to enterprises** while maintaining **security and privacy of data**.
* 🔒 The startup has already made significant inroads, being the **most deployed in the US government** and powering systems for companies like Saudi Aramco, facilitating **secure AI model training** within their infrastructure.
* 💡 To expand market share, the focus is on **democratizing access to this technology** by reducing costs and power requirements, enabling deployment in **compact spaces** and making it available to a **broader range of customers**.
* 📈 With strong VC backing and a commitment to restructuring power needs, the startup aims to outpace competitors by offering solutions that deliver **higher performance and lower energy consumption** than traditional GPU technologies.

You're kind of an invidious competitor,
upstart. And I just want to articulate for our
audience what it is that you do. Yeah.
Now, thanks for having us. So some of us started as a startup
coming out of Stanford University in

part to really focus on efficient
computing. And so as we're seeing a video scale and
you see all the other great numbers that that people are talking about within
video or we're now seeing is what happens when the world wants to go into
production and go into scale.

And one of the things that frankly,
we're not talking enough about is the fact that power and energy is going to
be a big bottleneck. People are talking about building
nuclear power plants and things like that.
Some innovator took the focus of how do

we actually take A.I.
at scale power efficiency and drive it down so that you can actually scale
without running into these limits that we start seeing with both companies and
countries running out of power and figuring out how to actually, you know,
provide enough energy to supply these

GPUs.
So your view is you can't like there's going to be a certain limitation on the
amount of energy efficiency that a company like NVIDIA is ever going to hit
with a GPU, whereas you offer what? Well, so today, for example, you know, a
single rack of a video is north of 100

kilowatts.
And just to give your audience a sense, that's 100, it's, it's a hundred homes
that it powers. And so if you think about somewhat of
technology, we're able to drive ten x a

performance at 1/10 the power. How now, because of a completely different architecture, as you know, with Nvidia, it's an architecture that's been around for several decades. And so we're trying to evolve this graphics GPU architecture into A.I. enabled now for inferencing.

There's so many new techniques that you can actually apply because you don't need to do all the heavy computing the training requires. And so you're able to drive a much, much lower footprint in terms of power, in terms of cost, in terms of space, which ultimately is required when you're

actually going to scale. And my belief is A.I. inferencing is going to be ten times more of a footprint than training. And so I think we're going to have to think about how to be energy efficient, how to be cost efficient before it gets to the scale that everybody gets to have

it. Can you, for our audience, that is not only an investing audience, but really a technologically savvy audience, describe to us what it is that's different that enables this higher energy efficiency within your particular type of technology, because it's not GPU

at all. What is it? Right. It's not graphics. And so it's something that was invented in research at Stanford University. It's called dataflow architecture. And what it does, it allows you to actually map the hardware in the way that a neural net wants to operate.

And so with NVIDIA today, you're having to break up these neural nets into these cores that exist inside the graphics chip, which turns out to be a lot of work, a lot of energy, a lot of power. Someone over with the data for

architecture allows you to avoid all of
that extra power that's being incurred when you actually break up these neural
networks and let it flow through the machine.
And so you're getting this order of magnitude improvements in power
efficiency, which ultimately allows you

to be able to deploy these in existing
datacenters, which, by the way, is one of the things I think is not talked
about when we actually acquire GPUs, because you also have to upgrade the
data centers where for our technology

going into existing datacenters with
existing power grids, with existing cooling systems, it becomes one of those
things that allows us to generate value for enterprises really, really quickly.
What enterprises are you already locked in with, how are you managing to sell this
narrative into corporates and federal

government?
Yeah, yeah. We're as a startup again and the idea is
at the top of the tier. We're a startup that's, you know, coming
in and providing disruptive technology. We're the most deployed in the US
government today as a startup.

You know, we're going to national labs
and places like that. And, and today we power also Saudi
Aramco's membrane, you know, so their internal GPT, this is a private equity
with they're trained into it and now

operating air gap within the company and
so companies they're worried about their data, their security, the privacy, can now
train these models on a platform completely securely and then deploy it
within their own infrastructure because you don't have to expose that data
outside your firewalls.

There are other startups in this space,
or indeed trying to offer more energy efficient options.
For example, also backed by Oracle, you yourself used to work at Oracle.
I'm interested as to what your next step is to really gain market share.
Is it about raising funds?

I know you got big VC backing, for
example. Yeah, I think it's a really good point.
My belief when we started the company back seven years ago is about access, one
democratized. I got to make it available to everybody
as your previous guest just said is

today investors really invest in large scale
only to a few customers. What we want to do, by collapsing the
power is to be able to make it available to everybody, to make it easily
available. And that means lower cost, much lower
power, and then able to deploy in very

tight spaces and not in the medium
gigawatt theater. And so some of we focused very much on
completely restructuring the power, that the power requirements of the air
infrastructure so they can deploy racks at performance that much higher than in
video.

Not less than a video, but much higher
than the video and much lower power.

Liang: Power and Energy Will Be the AI Bottleneck

Welcome to the Bloomberg Technology YouTube channel. Subscribe for original series and mini-docs on game-changing technologies, profiles on innovators, live conversations with experts, and breaking technology news & analysis from Bloomberg TV.

Bloomberg Technology

Company origins at Stanford University, focusing on efficient computing.

Addressing the energy bottleneck in large-scale video production and AI.

Highlighting the growing concern about power consumption in AI and the need for efficient solutions.

Focus on achieving AI at scale with significantly improved power efficiency.

Technological Advantages and Energy Efficiency

Comparison of power consumption: a single rack of video consumes power equivalent to 100 homes.

Achieving 10x performance with 1/10 the power compared to existing GPU technology.

Utilizing a novel dataflow architecture, unlike the decades-old GPU architecture.

Lower power, cost, and space footprint due to optimized architecture for AI inferencing.

Anticipation of AI inferencing requiring significantly more resources than training.

Introduction of dataflow architecture, invented at Stanford, allowing hardware mapping to neural net operation.

Comparison with NVIDIA's approach, highlighting the inefficiency of breaking up neural nets across cores.

Dataflow architecture avoids extra power consumption from breaking up neural networks.

Advantage of deploying in existing data centers without upgrades.

Significant deployment in US government and national labs.

Examples of current clients: US government, Saudi Aramco.

Focus on secure data handling and on-premise deployment to address data privacy concerns.

Addressing data security and privacy concerns for clients.

Discussion on gaining market share and competition in the energy-efficient AI space.

Company's focus on democratizing access to efficient AI through lower cost and power consumption.

Goal of making AI accessible to everyone by reducing power requirements and deployment size.

Focus on restructuring power infrastructure for higher performance and lower power consumption compared to existing solutions.

Achieving significantly higher performance with lower power consumption than competitors.