this is going to be an intro course to everything that you need to know about Rag retrieval augmented generation rag is a really misunderstood topic so I'm really excited to share just a little bit of knowledge with you about it and thanks to pine cone for not only sponsoring this video but being my exclusive Channel partner and pine cone has an incredible Vector database product which is the core of how rag works and I'll tell you more about that as we go on through this video one of the major misunderstandings about Rag and large language models in general is how to give large language models additional knowledge a lot of people think that is what fine-tuning is for and when you fine-tune a model you give it a bunch of additional information and then it's just going to know it from then on out but that's not really what fine tuning is for and in fact what I found is nine out of 10 times when you think you need fine tuning what you actually need is retrieval augmented generation so rather than doing the more

complicated task of fine-tuning a model, which is really for informing the model how you want it to respond to you in terms of tone. Giving it additional knowledge is as simple as using RAG. So first, let's start with an overview of RAG. As I mentioned, RAG means retrieval augmented generation, basically giving large language models an external source of information to augment your prompt. I like to think about RAG in two ways. One, it's an incredibly fast and efficient way to give your large language model additional knowledge, and it's an incredible way to give your large language model long-term memory, which they do not have out of the box. So you can think of large language models as being kind of frozen in time. Once they're done training, they do not get any additional information unless you give it to them. And there are a few ways to do that. Obviously, you can provide additional knowledge to your large language model through the prompt itself, and this is fine. But as soon as you start needing to do things even at a little bit of scale, that method breaks.

down quickly, and there's a reason for that. It's because the context window of a large language model is limited. So what's a context window? A context window is the number of words or tokens that you can give a large language model in your prompt, combined with the number of words or tokens that it is outputting as a response back to you. And those are typically limited. For example, GPT-3 by default has an 8,000 token context window. Even the frontier models like GPT-40 have a 128,000 token context window. That may seem like a lot, but you actually use up that context window really quickly, especially when you're talking about giving a ton of additional knowledge to a large language model or storing a lot of memory from that large language model. And not only that, as context windows continue to increase, which they are, they actually tend to be really inefficient and really costly as a method to continuously give new knowledge to a large language model. And that's why we keep turning to retrieval-augmented generation. So let's think

about an example, let's say we're building a chatbot for customer service, and that chatbot needs to store the conversation between the customer and itself forever. That'll allow it to continue to get more personal and understand that user better and better over time. Now, if we did not have RAG, what we would do is every single time there was a back and forth between the customer and the bot, we would have to feed that back into the prompt so that the large language model knew the history of the conversation. And you can imagine after hundreds or thousands of back and forths, all of a sudden you're using that entire context window really quickly every single time you submit a prompt. And here's the thing, all of that history might not be relevant to what you're trying to get the large language model to respond to in that moment. Let's use another example. Let's say we have internal company documents that we want to have the large language model have knowledge of. First of all, when this large language model, let's again just

use gp4 when we're using it. It has no knowledge of my internal company documents, and that's because that model wasn't trained on any of my internal company documents, which it shouldn't be, but I want to give it access to that knowledge. So, if we were to take all of those documents and put it directly into the prompt every single time, along with whatever question we had of those documents, that may work. But as you can imagine, the majority of that prompt window is going to be used up every single time, and more than likely, you're going to run out of that prompt window eventually if you have enough internal documents. So, what do we do with that? That is when we use Rag. And as I mentioned earlier, large language models are frozen in time. So, if something new in the world happened and we want to give that large language model knowledge about that event or whatever happened, Rag is a really great way to do that. All right, so let me talk about Rag from the highest level. What actually is it?

taking information let's say a document, and you're putting it in external storage. I'll talk about what that actually looks like in a bit, and then you're giving that large language model the ability to query or ask questions of that document. You can combine that with your prompt, so let's use a simple example. Let's say Tesla came out with their new earnings report. Now, if we're using GPT-4 and its knowledge cut off was months ago, it would have no knowledge of that new quarterly report from Tesla. But what we can do is we can take that quarterly report, we can store it in our rag database, and then anytime that we ask a question about Tesla and their recent earnings, we can actually take an intermediary step which goes and gets the relevant information from that document and appends it to our prompt. So, our prompt would look something like, Tell me what Tesla's earnings were in this latest quarter, and along with that, under that prompt and included in the prompt, I should say, is the relevant information from the 10K document about

their recent earnings. And now, instead of taking that 10K document and putting it in the prompt every single time, we are only taking the relevant information based on what we need in that moment. Now, this is a very simple example, but imagine if you wanted the 10K documents from the top Fortune 500 companies. All of a sudden, you can see you're not going to be including that every single time in the prompt. You wouldn't want to either because Tesla's Revenue has nothing to do with Apple's Revenue, has nothing to do with Meta's revenue, and so on. You're asking about Tesla's Revenue, so really the only thing that we should append to our prompt is the information, the knowledge about Tesla's Revenue. So let me give you another really specific example. So here's a workflow without Rag, and this is from the Pinecone learning documentation. So we have a generative AI search, so just asking a question to a large language model. And here's the question, how do I turn off the automatic reverse breaking on the Volvo?

XC60 now that question gets prompted to the model. Let's say chat GPT. And since it doesn't have that exact information, it's going to hallucinate, meaning it's going to make something up. So press the settings button on the center console or the steering wheel. Use the buttons or the touch screen to navigate and so on. But this isn't actually what is happening. So there is a much better way of doing this. So now let's look at what we need to do to actually prepare our retrieval augmented generation for that same question. So we take all of the Volvo user's manual. And here's the thing, our model doesn't have access to this currently. So that is why it hallucinated. So we're going to take all of them. We're going to send it to an embedding model. And I'll talk about what that is in a second. And then we're going to take that embedding and put it into a vector database. Now let me break all of this down for you. So the Volvo user's manual

is just exactly that. Let's say it's PDFs or a text document, just the entire manual for the Volvo cars. Then we send it to an embedding model, which converts it from text format to embedding format. An embedding format is just a series of numbers representing those words. And in the simplest terms, that embedding places that natural language somewhere on a multi-dimensional graph. And in this example, we're just using two dimensions. So here, it would place it at one. And then if we have a bunch of those, right? So here is an example. We have a bunch of different points in this vector space. What ends up happening is words, terms, entire phrases that are similar to each other start to end up close to each other, quote unquote, close to each other in this vector space. And it's a really interesting and awesome way to see if words and phrases and terms are related to each other. So let's look here. So imagine this is part of the vector space and we can see all of these words that are located near each other so connects.

occupies, operates, operated, bounded, elevation. All of these words are near each other in this vector space. And then what we do is we have now this intermediary step before we actually prompt the large language model. So we have the user who sends a query, again the same one about the Volvo. It first goes to the embedding model. The query itself gets converted into an embedding. Then we query the vector database and we say, What's similar to this question about the Volvo? And it grabs all the similar relevant data. And then we can combine the original query plus all of this additional context or knowledge, send it to the model, and then we actually get a relevant, accurate answer. The driver can choose to deactivate Auto Brake with rear Auto Brake and cross-traffic alert. The warning signal can be deactivated separately and so on. And so now we have non-hallucinated accurate information because we were able to give our large language model an external knowledge source. And RAG is not only powerful with large language models, but when you start to abstract things and

use agents. It becomes even more powerful, and RAG is one of the main tools that agents use to have all of this additional knowledge. So let me show you this example. So what we're looking at is we have a question. Aside from the Apple remote, what other device can control the program Apple remote was originally designed to interact with? Now if you ask the LLM directly, it's going to give you a very basic direct answer, iPod. Now if we use agents with the power of RAG, we can allow the agents to iteratively come up with a plan, do the research, incorporate external knowledge sources, and then come up with an even better answer. So let's look what that looks like, thought one. And you can just think about each of these thoughts as maybe different agents, but they can be structured in different ways. I need to search Apple remote and find the program it was used for. So search Apple remote, and this might be an external knowledge source that details how the Apple remote is used or what it's used for. Observation one: the Apple remote

designed to control the front row Media Center then thought to I need to search the front row and find other devices that control it. So now search front row documentation then front row is the software was controlled by an apple remote or the keyboard functions then another one front row is controlled by an apple remote or the keyboard function keys. I now have the answer answer keyboard function keys so it was able to use a lot of different sophisticated tactics with this external knowledge source and now let's take a step back and look at what's actually happening again in a little bit more detail. Remember we talked about putting the embeddings in a vector space well that's exactly what pine cone is built for but instead of one point with two Dimensions it can have billions or even trillions of points and tons of different dimensions that is why pine cone is so good and that is why Vector storage is so good it is lightning fast and you use natural language to query against it. Now if this sounds familiar to just search that's typically how search is done so

if you're searching for something, you're searching in natural language. You're typing out your query in natural language, and then your query is plotted on that Vector space, like I mentioned, and it looks for everything that is relevant to your query, relevant meaning close in the vector space, and then it returns it. And that's how search works, and that's also how retrieval augmented generation works, and Pinecone makes it super easy to do that. In fact, you really don't need to know anything about how the stuff actually works as a developer to actually go and build something with it. And that was the case for me when I first started. I didn't really understand what RAG was or what a vector database was, but I was still able to build something with it. And all you do is you take your data, you send it to an embedding model (and there are plenty to choose from), then you send it to Pinecone, and they'll store it for you. And then when you're ready, create a query against it, you send them your query in

embedding format and they will return the results for you. And there's a ton of different options. And what I will say is pine cone is known for scale. It is super fast and incredibly high scale efficient. And by the way, if you want to see me do a full tutorial about how to actually set all of this up from provisioning your pine cone account to taking your documents, converting them to embeddings, storing them in the vector space, let me know in the comments. I'm happy to do that. It's actually a lot simpler than you think. So that's the basics of rag for today. If you want to see me do a tutorial using pine cone, let me know in the comments. If you want a deeper dive into rag, also let me know in the comments. And again, a very special thank you to pine cone for sponsoring this video, for partnering with me on the channel. And I'm going to drop all of the links to Pine Cone and everything else you need to know about rag in the description below. If you enjoyed this video, please consider giving a like.

Subscribe, and I'll see you in the next one.

How to give AI "Memory" - Intro to RAG (Retrieval Augmented Generation)

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **