Caltech Commencement Address 2024:  NVIDIA CEO Jensen Huang on AI, Accelerated Computing, and Finding Your Life's Work

NVIDIA's CEO shares insights on navigating the AI revolution, emphasizing resilience and dedication to one's craft.

### Jensen Huang's Caltech Commencement Speech:  A Call to Embrace the AI Revolution
* 🎓 Jensen Huang, CEO of NVIDIA, addressed the Caltech class of 2024, highlighting the transformative power of **AI** and **accelerated computing**.
* 👨‍🎓 He emphasized the importance of perseverance, resilience, and the ability to overcome setbacks, drawing from his own experiences at NVIDIA.
* 💡 Huang recounted NVIDIA's journey, from its early days to its pivotal role in the AI revolution, sparked by the **AlexNet** breakthrough in 2012.
* 🤖 He discussed the shift from instruction-driven to **intention-driven computing**, where AI understands and executes tasks like humans.
* 🏭  Huang introduced the concept of "**AI factories**,"  new data centers producing "intelligence tokens,"  a new industry fueled by AI.
* 🧑‍🔧  He shared the story of a meticulous gardener in Kyoto, emphasizing dedication to one's craft and the importance of prioritizing life's work.
* 🚀  Huang encouraged graduates to embrace the uncertainties of the future, viewing setbacks as opportunities for growth and innovation.
* 🏆 He concluded by urging graduates to find a craft they are passionate about and dedicate their lives to perfecting it,  emphasizing that "**pain and suffering**... are the ultimate superpowers".

It really makes me cringe listening to all of that. Thank you for that kind introduction, but I hate hearing about myself.

And just as a show, well, maybe if you could just applaud. How many of you know who NVIDIA is?

And how many of you know what a GPU is? Okay, good, I don't have to change my speech.

Ladies and gentlemen, President Rosenbaum, esteemed faculty members, distinguished guests, proud parents,

and above all, the 2024 graduating class of Caltech. This is a really happy day for you guys.

You got to look more excited. You know you're graduating from Caltech. This is the school of the great Richard Feynman, Linus Pauling,

and someone who's very influential to me and our industry, Carver Mead. Yeah, this is a very big deal.

Today is a day of immense pride and joy. It is a dream come true for all of you,

but not just for you, because your parents and families have made countless sacrifices to see you reach this milestone.

So let's take this moment and congratulate them, thank them, and let them know you love them.

You don't want to forget that because you don't know how long you'll be living at home.

You want to be super grateful today. As a proud parent, I really loved it when my kids didn't move out,

and it was great to see them every day, but now they've moved out, it makes me sad.

So hopefully you guys get to spend some time with your parents. Your journey here is a testament to your character,

determination, willingness to make sacrifices for your dreams, and you should be proud. The ability to make sacrifices, endure pain and suffering,

You will need these qualities in life. You and I share some things in common. First, both chief scientists at NVIDIA were from Caltech.

And one of the reasons why I'm giving this speech today is because I'm recruiting. And so I want to tell you that NVIDIA is a really great company,

I'm a very nice boss, universally loved; come work at NVIDIA. You and I share a passion for science and engineering,

and although we're separated by about 40 years, we are both at the peaks of our careers.

For all of you who have been paying attention to NVIDIA and myself, you know what I mean.

It's just that in your case, you'll have many, many more peaks to climb. I just hope that today is not my peak.

Not the peak. And so I'm working as hard as ever to make sure that I have many, many more peaks ahead.

Last year, I was honored to give the commencement address at Taiwan University, and I shared several stories about NVIDIA's journey

and the lessons that we learned that might be valuable for graduates. I have to admit that I don't love giving advice,

especially to other people's children. And so my advice today will largely be disguised in some stories that I've enjoyed,

and some life experiences that I've enjoyed. I'm the longest-running tech CEO in the world today, I believe.

Over the course of 31 years, I managed not to go out of business, not to get bored, and not to get fired.

And so I have the great privilege of enjoying a lot of life's experiences, starting from creating NVIDIA, from nothing to what it is today.

And so I spoke about the long road of creating CUDA, a programming model. The programming model that we dedicated over 20 years to inventing,

and that is revolutionizing computing today. I spoke about a very, quite public, canceled Sega game console project we worked on,

and where intellectual honesty, something that I know Richard Feynman cared very deeply about and spoke quite often about,

where intellectual honesty and humility saved our company. And how a strategic retreat was one of our best strategies.

All of these are counterintuitive lessons that I spoke about at the commencement. But I encouraged the graduates to engage with AI,

the most consequential technology of our time. And I'll speak a little bit more about that later,

but all of you know about AI. It's hard not to be immersed in it and surrounded by it,

and a great deal of discussion about it. And of course, I hope that all of you are using it and playing with it,

with surprising results, some magical, some disappointing, and some surprising. But you have to enjoy it, you have to engage with it,

because it's advancing so quickly. It is the only technology that I've known that is advancing on multiple exponentials simultaneously.

And so the technology is changing very, very quickly. So I advised the students at the Taiwan University to run, don't walk,

and engage the AI revolution. And yet, one year later, it's incredible how much it's changed.

And so today, what I wanted to do is share with you my perspective, from my vantage point, on some of the important things that are happening

that you're graduating into. And these are extraordinary things that are happening that you should have an intuitive understanding of,

because it's going to matter to you, it's going to matter to the industry. And hopefully, you take advantage of the opportunities ahead of you.

The computer industry is transforming from its foundations, literally from the ground up. Everything is changing from the ground up.

And across every layer, and soon, every industry will also be transformed. And the reason for that is quite obvious,

because computers today are the single most important instrument of knowledge. And it's foundational to every single industry and every field of science.

If we are transforming the computer so profoundly, it will, of course, have implications in every industry.

And I'll talk about that in just a little bit. And as you enter industry, it's important you know what's happening.

Modern computing traces back to the IBM System/360. That was the architecture manual that I learned from.

It's an architecture manual that you don't need to learn from.  Much better documentation and better descriptions of computers and architecture

have been presented since. But the System/360 was incredibly important at its time, and in fact, the basic ideas of the System/360, the architecture of it,

the principal ideas and architecture and strategy of the System/360 are still governing the computer industry today.

And it was introduced a year after my birth. In the 80s, I was among the first generation of VLSI engineers

who learned to design chips from Mead and Conway's landmark textbook. And I'm not sure if it's still being taught here.

It should be in the introduction to VLSI systems. Based on Carver Mead's pioneering work here at Caltech

on chip design methodologies and a textbook that revolutionized IC design. And it enabled our generation to design supergiant chips and ultimately the CPU.

The CPU led to exponential growth in computing. The performance, the incredible technology advances, that's called Moore's Law,

fueled the information technology revolution. The industrial revolution that we are part of, that my generation was part of,

saw the mass production of something the world had never seen before. The mass production of something that was invisible, easy to copy,

the mass production of software. And it led to a $3 trillion industry. When I sat where you sat, the IT industry was minuscule.

And the concept that you could make money selling software was a fantasy. And yet today, it's one of the most important commodities,

most important technologies and product creations that our industry produces. However, the limits of Moore's Law scaling, of transistor scaling,

and instruction-level parallelism have slowed CPU performance. And the slowed CPU performance gains are happening at a time when computing demand continues to grow exponentially.

This exponentially growing gap between the demand for computing and the capabilities of computers, if not addressed, will cause computing energy consumption and cost inflation,

which would eventually stifle every industry. We see very clear signs of computing inflation as we speak. And after two decades of advancing NVIDIA's CUDA,

NVIDIA's accelerated computing offers a path forward. That's the reason why I'm here. Because finally, the industry realized

the incredible effectiveness of accelerated computing at precisely the time that we're witnessing computing inflation after several decades.

By offloading time-consuming algorithms to a GPU that specializes in parallel processing, we routinely achieve 10, 100, sometimes 1,000-fold speedups,

saving money, cost, and energy. We now accelerate application domains from computer graphics, ray tracing, of course, to gene sequencing, scientific computing,

astronomy, quantum circuit simulations, SQL data processing, and even pandas data science. Accelerated computing has reached a tipping point.

That is our first great contribution to the computer industry, our first great contribution to society: accelerated computing.

It now gives us a path forward for sustainable computing where cost will continue to decline as computing requirements continue to grow.

A hundred-fold improvement in time, cost, or energy savings, which accelerated computing enabled, surely would trigger

a new development somewhere else. We just didn't know what it was until deep learning came to our consciousness.

A whole new world of computing emerged. Jeff Hinton, Alex Krizhevsky, and Ilya Sutskever

Using NVIDIA CUDA GPUs to train AlexNet, we shocked the computer vision community by winning the 2012 ImageNet Challenge.

This was a pivotal moment, the big bang of deep learning, marking the beginning of the AI revolution.

Our decisions after AlexNet transformed our company—and likely everything else—in a way worth noting.

We saw deep learning's potential and believed, based on principled thinking and our analysis of its scalability,

that it could learn other valuable functions. We thought deep learning might be a universal function learner, solving problems difficult

or impossible to express using fundamental first principles.  So, we thought,

this is a technology we must prioritize, as its limits are potentially constrained only by model and data scale.

However, challenges existed in 2012. How could we explore deep learning's limits without building massive GPU clusters?

We were a small company; building such clusters could cost hundreds of millions of dollars.  And if we didn't,

there was no guarantee of effectiveness at scale.  No one knew how far deep learning could scale.

And if we didn't build it, we'd never know.  It was a 'build it and they will come' situation,

but our logic was: if we don't build it, they can't come. So, based on our principled beliefs and analysis,

we dedicated ourselves to it. We reached a point where we believed it would be so effective,

and when the company believes something, we act on it. So we dove deep into deep learning.

And over the next decade, we systematically reinvented everything. We reinvented every computing layer, starting with the GPU itself.

The invention of the modern GPU, which is very different than the GPUs of the past that we invented,

and we went on to invent just about every other aspect of computing: the interconnects, the systems,

the networking, and of course, the software. We invested billions. We invested billions into the unknown.

Thousands of engineers for a decade worked on deep learning, advancing and scaling it without really knowing how far we could take the technology.

We invested billions. And we designed and built supercomputers to explore the limits of deep learning and AI.

And in 2016, we announced DGX-1, our first AI supercomputer, and I delivered the first one to a startup in San Francisco,

a startup nobody knew anything about—a group of friends of mine who were working on artificial intelligence,

a company called OpenAI. In 2022, 10 years after AlexNet, and about a million-fold increase in computing power later,

a million-fold. Imagine what it would be like if your laptop were a million times more capable?

Jensen Huang’s Speech At CalTech｜黃仁勳加州理工演講

New SciTech 新科技

Huang expresses discomfort with self-congratulatory introductions.

He gauges audience familiarity with NVIDIA and GPUs.

He formally addresses the Caltech graduating class and guests.

He mentions influential Caltech alumni, highlighting the institution's prestige.

He acknowledges the sacrifices of parents and families.

Huang emphasizes the importance of perseverance and sacrifice.

He highlights Caltech's connection to NVIDIA and subtly recruits graduates.

He mentions his previous commencement speech and his approach to giving advice.

He reflects on his long tenure as a tech CEO and the lessons learned.

He discusses NVIDIA's work on CUDA and a canceled Sega project.

Huang discusses the fundamental changes occurring in the computer industry.

He explains that computers are the most important instrument of knowledge.

He traces the history of modern computing, mentioning the IBM System/360.

He discusses the impact of Moore's Law and the limitations of CPU performance.

He introduces accelerated computing as a solution to growing computing demands.

He describes the impact of AlexNet and the beginning of the AI revolution.

He discusses the challenges and risks of investing in deep learning.

He explains NVIDIA's significant investment in deep learning and AI.

He highlights the launch of ChatGPT and the mainstreaming of AI.

He describes the shift from instruction-driven to intention-driven computing.

The Future of AI: Robotics and Lessons Learned

He discusses the next wave of AI, focusing on robotics.

He recounts NVIDIA's experiences with setbacks and pivots in various markets.

He emphasizes the importance of resilience and adapting to challenges.

He shares a story about a gardener in Kyoto, highlighting dedication and perseverance.

He describes his daily routine, prioritizing high-impact tasks.

Concluding Remarks and Advice to Graduates

He congratulates the graduating class and offers final words of wisdom.

He encourages graduates to believe in unconventional ideas and persevere through setbacks.

He identifies resilience and the ability to overcome challenges as key strengths.

He advises graduates to find a craft to dedicate their lives to.

He emphasizes the importance of prioritizing life and having plenty of time.