AWS re:Invent 2024 - Monday Night Live with Peter DeSantis

The official YouTube channel for AWS Events and Webinars. Amazon Web Services (AWS) hosts events, both online and in-person, bringing the cloud computing community together to connect, collaborate, and learn from AWS experts.

AWS re:Invent is a learning conference for the global cloud computing community. The event features keynote announcements, training and certification opportunities, access to technical sessions, deep-dives with AWS leaders, engaging with sponsors, and so much more.

AWS re:Inforce is a two day conference focused on cloud security. Attendees can expect two days of learning, with sessions on security, identity, and compliance. 

AWS Summits is a series of events that bring the cloud computing community together to connect, collaborate, and learn about AWS. 

AWSome Day Online Conference is a free, online training event that will provide a step-by-step introduction to the core AWS services for compute, storage, databases, and networking.


AWS Events

Peter DeSantis, SVP of Utility Computing at AWS, welcomes the audience to re:Invent 2024.

DeSantis introduces Monday Night Live and apologizes for the lack of IPA beer.

The focus will be on technical innovations, potentially including a new product announcement.

Emphasis on the 'how' of AWS's cloud computing capabilities and their design.

Discussion of AWS's unique approach to delivering cloud computing capabilities.

AWS's Unique Approach: The Analogy of Trees

DeSantis explains his use of an AI assistant to develop the presentation's structure.

Initial analogy of an iceberg is rejected; a space flight analogy is considered but ultimately discarded.

The analogy of trees is introduced: trees represent major technical investments (custom silicon, hypervisors, database technologies).

Focus shifts to the 'roots' of AWS's success, starting with the 'taproot'—leadership's attention to detail.

Importance of leadership's involvement in details for fast decision-making and issue prevention.

AWS Innovation Across the Stack: Buttress Roots and the Wood Wide Web

The analogy shifts to buttress roots, representing AWS's innovation across the entire stack.

AWS's deep investment across various components enables unique capabilities.

The 'Wood Wide Web' analogy is introduced, representing AWS's interconnected culture.

The symbiotic relationship between trees and fungi illustrates the importance of AWS's culture.

AWS's unique culture, built through leadership principles and mechanisms, is highlighted as a key differentiator.

AWS Compute Networking Innovations: Graviton Processors

Dave Brown discusses his AWS journey and the development of EC2.

The evolution of Graviton processors is detailed, emphasizing their performance improvements across generations.

Comparison of microbenchmarks versus real-world workload performance for Graviton processors.

AWS's internal migration to Graviton and its impact on services like Aurora, DynamoDB, and Redshift.

Transition to the AWS Nitro System and its role in enhancing performance and security.

Explanation of the AWS Nitro System's impact on server architecture and security.

Focus on cryptographic attestation and its role in ensuring hardware supply chain integrity.

Detailed explanation of the boot process and the role of the Nitro chip in securing it.

Extension of attestation to the Graviton4 processor, creating an interlocking web of trust.

Discussion shifts to the challenges and solutions related to storage dynamics.

Evolution of hard drive capacity and cost per terabyte.

Explanation of the components of storage systems (front-end fleet, index/mapping service, storage media layer).

Discussion of the traditional head node architecture and its limitations.

Description of the Barge project and the lessons learned from its limitations.

Introduction of the concept of disaggregated storage using Nitro cards.

Description of the disaggregated storage architecture and its benefits.

Focus shifts to AI workloads (model training and inference).

Distinction between scale-out and scale-up workloads in the context of AI.

Discussion of the growth of AI models and the impact of scaling laws.

Explanation of how large AI models are trained and the need for parallelization.

The scale-up and scale-out challenges of AI infrastructure are highlighted.

Trainium 2 and the Most Powerful AI Server

Discussion of building the most powerful AI server, focusing on the Trainium 2 chip.

Explanation of the engineering limits related to chip size and packaging.

Description of the Trainium 2 package and its components.

Discussion of power delivery and voltage drop optimization in Trainium 2.

Description of the Trainium 2 server's architecture and its performance capabilities.

NeuronLink, Ultra Servers, and AI Inference Optimization

Introduction of NeuronLink interconnect technology and ultra servers.

Description of an ultra server's capabilities and its suitability for trillion-parameter models.

Discussion of AI inference workloads (input encoding and token generation).

Announcement of latency-optimized options for Amazon Bedrock and its performance improvements.

Announcement of a latency-optimized version of the Claude 3.5 model.

Anthropic's Collaboration with AWS and Project Rainier

Tom Brown, co-founder of Anthropic, discusses their collaboration with AWS.

Discussion of Claude 3.5 and its latency-optimized version on Trainium 2.

Importance of optimizing the work sequencing for Trainium 2 to maximize performance.

Trainium's ability to log instruction timing for improved kernel development is highlighted.

Announcement of Project Rainier, a new Amazon cluster with hundreds of thousands of Trainium 2 chips.

Focus shifts to the scale-out aspect of AI infrastructure.

Discussion of the need for an elastic AI-optimized network.

Description of the 10p10u network fabric and its capabilities.

Discussion of innovations to improve network installation and reduce errors.

Focus on improving network reliability through innovations in optical links and routing protocols.

Summary of the key innovations discussed and their benefits for customers.

DeSantis concludes the presentation and thanks the audience.