hello and welcome to best practices building generative AI applications on AWS. My name is Dan Stair. I am an analytic specialist Solutions architect, and this video series was built in collaboration with my co-workers Hara Tataparthy and Felix Huthmacher. Our objective is to provide a one-stop shop for you to learn the essentials of building a generative AI application on AWS. In our role, we assist many customers of different sizes in realizing their vision for generative AI applications. Since the technology is so new, we have seen that many customers lack the proper level of understanding needed to build a generative AI application. This video series starts at an early stage of learning how to choose a model and takes you all the way through deploying and monitoring a large language model in production. This is the first video in a four-video series. In this video, we will provide an overview of model selection and how to choose the right large language model for your use case. In video two, we will show a demo of model selection. In video three, we will show

how to improve the performance of your large language model. Once it has been chosen, and finally, in video 4, we will review best practices for large language model operations and how to deploy and monitor your large language model in production. But to start, what is a large language model? A large language model can be defined as a pre-trained deep learning model, which can generate human-like text responses. Large language models take input text, called a prompt, and predict the next word or token in the sequence. For example, in this example, we have the input tokens the students opened there, and the large language model is considering one of several next tokens. It believes that there is a 99.6% chance that the correct next token is books, and there is an 88.2% chance that the correct next token is laptops. Once it is chosen the correct next token of books, it may continue to iterate on the sequence. So now the input text is the student open their books, and it may choose the next token in the sequence to make the sentence the students opened their books.

quickly. It will continue to iterate in order to generate more output until it is done. But how can you get started on your journey with large language models, and what does a typical large language model journey look like? We recommend that you start by defining your use case. So, what would a good end result look like? Then narrow down the set of models to evaluate based on the set of criteria. For example, the size of the model, the end-user license agreement of the model, or and the purpose that the model was built for. Once you have narrowed down the set of models you are considering, we recommend that you employ one or more techniques to improve model performance, including prompt engineering, retrieval, augmented generation, and or reinforcement learning with feedback. Then you will need to move on to evaluate the results of your model. Once you have improved the performance on the model, in the next video, we will cover recommended ways, which are quantitative, automated, and repeatable, to help you evaluate your large language model's performance. You may need to iterate on

these performance improvements steps depending on how your evaluation metrics compare to your Target. And finally, once you've selected and optimized a large language model for your use case, we will show you how to deploy and model your LLM in production. So to get started on this journey, what are some of the building blocks that you will need? One useful tool is embedding vectors. Large language models store text data as numbers. In order to store large amounts of information or quickly search for related information, you will need to convert text into vectors. In this example, the text the sky is blue is converted into a vector, which is basically a set of numbers. Once you have generated your vectors, you will need to store them in a vector database. And finally, once you have provided your model with the data it needs, you will need to consider several patterns commonly used to improve performance. These include prompt engineering or using different prompts to optimize responses the model gives, retrieval augmented generation in which you augment the model's knowledge with a vector database, or you may choose to find.

tunar model, which involves providing the model with additional training data, so that it learns about your use case specifically. Or you may choose to train a large language model from scratch, which is the most expensive in terms of time and compute cost, but can make sense for highly specialized use cases. Now that we have covered some building blocks, we will cover, at a high level, the recommended steps to select the right model for your use case. Cas large language models do have some overlap, but many large language models are purpose-built to be good at tasks such as summarization, language translation, image creation, etc. So be sure to work backwards from your use case to filter for the appropriate large language models. Keep in mind that there are a number of model leaderboards out there, including Stanford Helm, Hugging Face's Open Large Language Model Leaderboard, and Hugging Face's Chatbot Leaderboard. But remember that these public benchmarks are not necessarily suitable for your data and your use case. Finally, keep in mind that larger large language models are not always better. Large language model, larger LLMs.

May cost more. They may incur higher latency, and the chinchilla scaling laws suggest that there's an optimal ratio of training data to model size. So, some use cases may actually be limited by the amount of training data available, in which case choosing a larger model will not deliver proportionately better results. There are several more tools available, which are important to look at when selecting your model. Many popular model hubs provide model cards, and these model cards often specify which use cases fit best with the model. For example, translation, summarization, or image generation, the input data type and input data volume for the model, the model size and number of parameters, and the end-user license agreement. Again, we recommend that you evaluate the model with your data for your use cases and finally arrive at the best model for your use case. So, once you've decided on a use case and chosen a model, you will need to get started using purpose-built tooling. AWS offers a rich ecosystem of services to help customers build and deploy large language models. One such service is SageMaker.

jump start, which is for customers who want to deploy a large language model onto dedicated hardware, offers a one-click deployment for a variety of large language models. These models are deployed on pre-tested instance types. Some models can be fine-tuned. And because this is accessed via SageMaker Studio, you can also use the full SageMaker ecosystem, which includes a feature store, GitHub integration, and SageMaker Pipelines. Another popular tool is Amazon Bedrock. Bedrock provides best-in-class foundation models with fully managed serverless inference and a simple API that is easy to build generative AI applications with. In addition to a user interface, both Bedrock and SageMaker Jump offer playgrounds. So, through the AWS Management Console, you can play with a model without having to actually deploy it first. And they can both be reached via APIs. So, Amazon Bedrock offers an API, and SageMaker inference endpoints also offer API-based inference. And you can access Bedrock using the AWS CLI in a programming language of your choice. For example, Python or JavaScript.

Etc. So, once you have chosen the right service to run your large language models on AWS, you will need to choose. You may need to choose a vector store. The top list consists of AWS-managed vector stores. Amazon OpenSearch is a popular data store for low-latency access to large data sets. It comes in both serverless and provision flavors and supports a purpose-built vector engine for Amazon OpenSearch serverless, which is in public preview as of today. Amazon Kendra is optimized for search and provides a rich connector ecosystem and also simplifies the process of creating embeddings for your vector store. The PG Vector plugin for Aurora Postgres and RDS Postgres are popular options for customers who are heavy users of Postgres. There are also several third-party tooling options, including Pinecone and Redis. These integrate with Amazon Bedrock via an integration feature called Bedrock agents. In summary, in this video, we discussed how to start your journey with large language models by picking a use case using popular available tools to.

choose large language models and gave an overview of the tools available on AWS to help on this journey now I will hand it over to Felix for our next video

Best Practices for GenAI applications on AWS – Model Selection - Part 1 | Amazon Web Services

The official YouTube channel for Amazon Web Services (AWS). 

Amazon Web Services offers a complete set of infrastructure and application services that enable you to run virtually everything in the cloud: from enterprise applications and big data projects to social games and mobile apps. Explore how millions of customers — including the fastest-growing startups, largest enterprises, and leading government agencies — are using AWS to lower costs, become more agile, and innovate faster. 
