Hey hi everyone, welcome to my YouTube channel. My name is Sun Savita and I'm back with another exciting and important video. So guys, this is the 13th video of this end-to-end Langra course and inside this video we're going to discuss about

the corrective RAG. Yes guys, so from this video onwards I'm going to start with a different different agentic RAG. Uh so in order to that first I will discuss about the corrective RAG, then I will discuss about the agent RAG, then adaptive RAG and then self RAG. Okay, so uh four to

five videos I'll be recording with respect to this particular topic and first of all let me do one thing, let me show you the syllabus. So guys, this is the entire syllabus of this uh Langra course. So inside the first slide I discuss all the fundamental concepts

related to the agents as you know if you're following to me then uh in the second slide basically I discuss all the basic and the necessary concept of the Lang graph. Then in the third slide guys I covered up to this common agentic pattern. Now only two main concepts are

remaining. The first one is the RAG, RAG with the Lang graph uh like C RAG, AAG and self RAG whatever I told you just now. And the third one basically sorry, the second one is multi-agentic system. So guys after completing this two topics

we'll be good, right? We'll be uh like good for creating any kind of application using the Lang graph because these two concepts are a very important concept and I've seen uh on the other YouTube like many videos you will find out related to this multi-agent and all but

no one is giving you the in-depth understanding and no one is giving you this type of understanding with respect to the Lang graph and all so guys if you want to uh understand the Lang graph LangChain and this AI application building and all everything so my

channel it might be one stop solution for all of you and here inside the video section you might get all the videos I created a different different playlist also so if you will go and check with the playlist section you will be getting the playlist with respect to each and

every topic whatever is required and I'm trying to create many more playlists in the future so yeah without wasting a time let's start with the topic so uh first of all guys uh let me explain you the problem statement and then I will come to the coding so I hope so you are

familiar with the syllabus you are uh familiar with the title of the video you are familiar with the like the content and all everything so here guys uh I written the problem statement on my Blackboard itself so first let's uh try to uh discuss about it and then step by

step one by one we'll try to implement it so uh first of all let me do one thing let me stop my video so that you can access the entire board I hope so it is visible uh now here see uh what's the topic name the topic uh name is uh like

a very simple it's a corrective RAG so let me write it down over here corrective RAG but before starting this corrective RAG we'll have to understand about the RAG guys uh this RAG actually uh I already covered so many videos on top of this

Right, so inside this particular video I will just give you a glimpse of it and if you want to understand in detail, you can check out my YouTube channel. Now, this is the architecture of RAG. Now, what's the full form of RAG? Tell me. Retrieval Augmented Generation, so

inside the RAG, uh there are three stages. So the first stage is called the data injection.  Means, we gather the data from somewhere, uh we divide it into multiple chunks, we perform the embedding out of it, and then we store it inside the database. So this particular process is called the data

injection. Now, whenever a user comes, so the user what it does, it asks any sort of a query. And here, this query basically it's converted into the embedding and we perform the similarity search or semantic search. Okay? And then we retrieve the uh ranked results. So ranked results or you can say the rank result

both, both are fine. Okay? And this uh refined data, okay, this retrieved data actually we pass to the LLM along with what? Along with my query and the required prompt. The context basically it is called the relevant data. Okay? The prompt basically I can give any sort of a

prompt for generating the output according to my requirement and query. Actually, it's an actual question. And then finally we get the output. So inside this RAG, we have three stages: first is this data injection, second is the retrieval, and the third is the generation part. I hope so this architecture is is clear

and it's a pretty straightforward architecture. Now, whenever we talk about the corrective RAG, okay, so what is the meaning of this corrective? So the corrective basically uh here what we are going to do, see, if my RAG is giving me a wrong answer, okay? So what we can do, we can

Correct that particular answer, we can correct that particular answer. So here, the corrective, corrective, basically it is uh representing this knowledge correction, knowledge correction. Okay. Now, how it is possible? How we are going to perform this knowledge correction and all everything, I will let you know. I have an architecture

with me. Okay. So guys, here uh this corrective RAG, basically we are going to implement using what? Using the L graph. Now, why I'm choosing this L graph? So L graph, basically see it gives you the flexibility uh so inside the L graph, what you can do, see you can define the

functions okay in the form of nodes and you can make a connectivity between these nodes, right? This is called edges. And guys, here you will find out one more thing that we can loop, okay? We, we can create a loop out of it, means let's say we are

going from the first node to the second node and the second node to the third node, then again if I want to come to the second node again, I can come over here and then again I can go back to the third node. Okay? So this is called a directed cyclic

graph, which I already explained to you. So this will help us for creating this corrective RAG. This will help us to create a corrective RAG, means let's say we are getting wrong information using the RAG, I can correct that particular information and inside my solution, from

where actually I'm going to correct my information? From the internet, okay? From the internet, if my RAG is not giving me any correct answer, what I will do? I will fetch the data from the internet and based on that, I will be generating my final output. I

Hopefully you got some sort of idea over here, but let me give you the comprehensive idea. So over here, guys, you can see this is the architecture of the corrective RAG. Okay, now let's try to understand about this corrective RAG, like how it is

working. So inside the corrective RAG, see what is happening. And this architecture, basically, we have defined or we have created using the LangChain graph. And the same architecture, we are going to implement. Okay, in some time, I will show you the code and all everything. So we have a question, guys. So whatever

question my user is asking, it will go to the retrieval node. Okay, it will retrieve the relevant document. Now my grader node, actually, it is checking, okay, this grader node, actually, it is checking whether my relevant document is correct, okay, or it is not correct.

Means, yes, means correct; no means not correct. Okay, so guys, let's say, uh, I'm giving my document to the grader node. So if there is not any irrelevant document, means everything is fine, so what I will do? I will go to this particular condition and finally, I'll be

generating my answer. Understood? Okay, now, guys, let's say if my document is an irrelevant document, means my document is not correct, so in that case, I will come to this particular side and I will be rewriting my query. So again, I will be

defining one node for that. I'll be rewriting my query, and then based on that, I will perform the web search. Okay, from the web, web, actually, from the internet, I will fetch the information and based on that, we are going to generate an answer. So here, guys, this web

Search is nothing, it is a way, okay? It is a way for correcting the information. So how are we going to correct the information if my retriever actually, after the data injection, the retriever, whatever retriever I'm going to create, okay, if this retriever is not able to

answer for anything, if this retriever is not able to answer anything, so what I will do, I will be going through the web and from this web actually we are going to fetch the correct answer and then we are generating the final answer, understood guys? So in my case, in my

scenario, in my solution, I see this web, okay, this web search, but guys in your case it could be anything, it could be any database you can fetch the correct answer from the database, it could be any other knowledge base, okay, wherever you kept the data over the cloud and all

somewhere, right? It could be any API, specific API, you can define any sort of logic, understood? So this is the complete architecture of the corrective R. Now again, let me give you the, uh, let me give you the like revision of it. So this is the, uh, implementation of the

Lang graph, which which we are going to implement as I told you guys, we are going to start with a question. So here I created a retriever node. This retriever node actually it is going to keep the retriever, okay, retriever basically which we are going to create from the vector

database, understood? I will be writing the entire code over here, don't worry. Then grade document, we are going to grade our document whether my document is correct or not, means yes or no, okay? Then, uh, basically after this yes or no, let's say my document is the correct

document what I will do. I will be going to this generate node and finally I'm generating my answer and I'm giving the output. Understood? But let's say my grid, okay, is no means I'm saying this document, actually, it's an irrelevant document, whatever document I'm going to capture.

Now, so it's an irrelevant document. In that case, what I will do if it is an irrelevant document? Again, I will transform my query, whatever query I was asking, and then I will perform the web search. And from the web search, I will be fetching the relevant data and then only

I'm going to generate my final answer. I hope so, guys. This architecture is clear to all of you. Now, if it is not clear, again, we'll try to understand it. Now, what is the crux of it? So, the crux is very simple: this great document, if my

relevant document is correct, okay, if my relevant documents document is correct, then I will directly generate the answer. If my relevant document is not correct, then I will take this approach: I will transform the query, I will perform the web search, and then I will generate the answer from the web itself. I hope so

this architecture is clear to all of you. Now, guys, whenever we are talking about this uh corrective RAG, okay, so here guys, this corrective RAG, actually, it's the implementation of the research paper. So, if you will go and check with Google itself, so just try to search about the

corrective RAG research papers, you will find out this particular research paper, right? So, here you will find out the complete detail, like how this, how people have implemented it. And it's a recent research, so see when they have published this research paper. In October itself, last month only. And here this

This research paper actually has a lot of potential, and see guys, this is the complete algorithm of it. So this is the complete algorithm which we are going to implement in the simplest way. The architecture which I showed you is an implementation of this entire algorithm.

If you have time, you can read it now. There are like a couple of more headings you can check. So how basically they are going to rectify the information, see, using web search? So inside the main research paper, they have used web search.

That's why, like, most of the solutions you will find out around web search. But guys, it could be anything. I hope so this thing is clear to all of you. Now, let's uh come to the implementation guide. So for practical guys, I'm going to create a file inside

the same folder. So uh like here, you can see I created one folder, different type, different type of RAG with a Lang graph. So inside this particular folder, I created this corrective RAG.ipynb, and this file you will get it over the GitHub itself. Now uh guys, see, I already

written a code in a refined manner. Okay, I wrote it by myself. I haven't copied and pasted it from somewhere else. So I will be copying and pasting over here inside this file, okay, the chunk of the code, and I will more focus to explain

the concept understood. So don't think that I'm going to copy and paste from somewhere. No, it's my code. I wrote it for all of you, and I'm not typing it because it's time-taking, or I might make a mistake, and all. So I, I'm just like a

copy and pasting from a notepad and I'll be keeping it over here and in the upcoming future I'll be creating a modular code and all uh and one more thing I would like to say the functionalities right so different different functions I will have to

implement over here as I like shown you so uh yes the code is going to be a little little leaner okay great now so uh here first of all we have to load the model and this time we are going to use the gni so I loaded the gni model and just a

second let me keep it over here so yeah loading the model a very simple command so loading the model great guys so let me load the model and here after loading the model what I'm going to do here I'm going to simply check it so for checking

the model uh here is the code so let me keep it over here this one see I'm going to check the code and yes it is working fine great yeah it is great now I written the question write a BAL about lent okay you can test it out with any sort of a

question I don't have any issue now the next thing what it is going to be so here I will create a retriever now okay uh means I will perform the data injection I will keep my data inside the vector database and I will perform the

similarity search on top of it and based on that I will be finding out the relevant document so uh here uh I'm going to make a heading so the heading is going to be very simple so here is what here is my retriever okay now uh

this is the complete code guys for the RAG and again I'm saying this RAG actually I cover extensively on my YouTube channel uh let me show you that playlist as well where actually I cover the complete playlist just a second so here I'm going to copy and paste this

code okay this is the RAG code now uh after uh creating this retriever what I will do see after creating a retriever I'll be creating a RAG chain okay again I'm saying guys this entire code has been written by me along with all the

comments and all before just before the class I finished it and here I am going to copy and paste just to save the time only now what I have I have this uh particular code so just a second and I will explain you that part don't worry

and here the next thing is going to be uh I'm just going to uh like a testing the chain okay so this is the testing and finally it's my testing code understood so on top of of this RAG itself I'll be performing the corrective

RAG using the Lang chain so uh now see guys this is the complete RAG pipeline which I have created if you'll go my YouTube channel you will find out so many playlist on top of the RAG you see inside the playlist section we have this Advanced RAG and this multi-modal RAG I

have this uh uh this uh complete RAG okay retrieval argument generation so please uh try to cover the RAG from this playlist itself and it will give you the comprehensive understanding of the RAG okay now coming to the next part so guys here is my complete RAG pipeline now let

Let me explain this part to you. Here is my statement: the import statement. This is my URL. From here, basically, we are going to collect the data. So, I collected some data. Let me show you what I, what we have inside this URL. So, if you will go and check, if you'll

copy and paste this URL, and if you'll paste it over the browser, you'll see this is the blog, actually. Okay, it's a, uh, blog post. So, you will find it out over this particular link, right? So, there are different, different blog posts. I see three

blog posts, and it is going to capture the entire text from here. Okay, it is related to the RAG and all this agent RAG. So, I'm, I'm going to keep the same one now. Uh, here, see what I have. I have this data. I am going to, like, fetch the

data using this B-based loader. Then here, I'm going to make a chunking. Then here, I'm going to store. So, this is the entire chunking code. Okay, uh, this chunk size and all everything. And here, I'm going to store it inside the ChromaDB, and

finally, I'm going to create a retriever from the vector database. Sorry, uh, I'm going to create a retriever on top of the vector database. Okay, this one. Now, let me create a retriever. If I want to check, so for checking, guys, uh, here is a

complete, uh, pipeline. Here is a complete chain. So, this is my prompt. Okay, here I kept my prompt. This is my LLM. Okay, uh, I kept my LLM, my Google's JAX LLM, and this is my output parser. So, whatever, like, output I will be getting, I'm parsing using this

output parser and I'll be getting my uh like final string okay so this I already explained to you inside the LangChain tutorial and all you can go and check out now prompt wise I will show you the prompt so this to this particular prompt we are passing only two things so the

first is a context okay here you can see context and the second is a question so uh let me execute this part and yes this is well and good and this is my prompt guys so inside the prompt context variable context and this question is my input variable and you can read the

complete prompt so once you will write swipe uh you will be able to read the entire prompt this one okay so based on this context actually I have to answer for my final question and this this that whatever has been written you can read it out from your hand okay and from

where I'm going to fetch this particular prompt I'm going to fetch it from The Hub okay so LangChain is having one Hub where actually they cap the entire prompt and all and from there itself we are going to fetch the prompt and this is my reaction now so let me execute it and

let me see whether it is working fine or not so I my question is very simple tell me about the agent memory and yes it is working very very well so I hope so this LangChain is clear to all of you now let's create a corrective rag on top of

it for creating a corrective rag guys let's create a skeleton of the code and then we'll create a different different function on top of it so for creating a skeleton of the code we required what we required the LangChain module so here what I'm doing I'm going to import this state

graph from the L graph, see this one, and now what I'm doing, I'm going to define my state. Okay, so see this is my state, and using this state, I'm going to define my workflow. Now you will be surprised over here, Sunny, why you are using all

this thing inside this state, because so far, I we we have just used the message. So guys, here the answer is very very simple: I want to flow all the values throughout the notes. It's not of like, basically, the thumb rule always, we have to define the

message, I can define anything. So here you learn the new concept, means I can flow anything throughout the node, and whenever it's required, when from the state whatever thing is required, I can patch those things. Understood? Great. Now let me do one thing, let me create a workflow.

Okay, so here is what, here is my workflow. Now let me create a different different notes. So guys, uh, these are my different different nodes. Here you can see retrieve grade document generation, transformer, transform query, and the web search. So this is going to my notes, and I think so

we haven't defined any sort of a function, it is giving me an error. Not an issue, we will create the function, and we'll look into it, uh, we'll look into the like the final output as well. Okay, so this is my note, and here is my ages.

Guys, so just just look into the ages also. So here I define the ages, so so, uh, guys see this is my node, retrieve grade document, generate, transform query, and web search. Understood? Now here is my ages. So first I will start from the retrieve, then I will pass it to the grid.

document okay now let's see if it is going to be passed then what I will do I will be generating the answer see here I mentioned the conditional statement if the grade document is going to be passed means my all the document is a relevant document in that case simply I will

return this General okay and then finally generate to end but let's see if this grader is giving me no it is saying okay your document is not a relevant one in that case what you will do you will transform the query you will pass it to the web search and then

web search basically it will fetch the output and then finally you will be getting finally actually you will be generating your answer and then only you will get output understood okay now what I'm doing here I'm going to compile my workflow so let me let me compile this

up entire workflow and finally I'll be displaying it okay so see guys this is my display now uh I think so this is a completely fine to all of you now let's do one thing let's try to create a function step by step because as I told

you uh inside the L graph this each and every node is nothing it's just a function okay let's create a different different function step by step one by one so I'm going to mark it down over here let's let's create let's create a

required functions okay now here I will write a code so there would be a like a four to five function so step by step one by one let me write all those function and that's why guys I told you this code is going to be little lengthier because of this function

itself so the first function is going to be retriever so this retriever actually what it will do it will retrieve the uh like uh relevant document based on the given question okay see this is my retriever function now the second function is going to be what the grader

document so whatever I'm going to retrieve okay just a second let me do one thing let me copy and paste it because this uh might be a little confusing but don't worry I will explain you everything so here whatever document I um getting so I will be grading those

particular document understood so the third thing basically generate so if uh like let's say if the grader is saying yes uh like everything is perfect no need to do the web search and all in that case I will simply generate the answer from this particular method okay

now the next one basically is going to be transform the query if uh basically grader is saying okay your question is not relevant to the generated document your question is not relevant to the generated document in that case I'll be transforming my query understood then

guys what I will do see here I will transform the query after that I will be fetching the information from the web and from web guys I'm going to use the TBL over here okay so for B I'm using the TBL and let me define the TBL and I'll

be fetching the web related I will be fetching the information from the web okay and then here is what here is my web search got it guys so this is going to be my web search just a second uh let me keep that functionality also and guys

see this code step by step one by one I created for all of you so don't think that I'm just copy and pasting from somewhere because I created it and I executed it in a precise way just to explain to you okay now web query is web

search is there and then here one more method would be there and this method is going to be very important uh which will help to the conditional statement that is decide to generate understood I hope so you got it now in between you will find out couple of uh this uh lines okay

this uh yellow lines it is saying query rewriter is not there and here you will find out one more retrieval grader is not there okay and here basically this two things I have to Define like this rag flow so let's try to Define this uh

query grader I will grade the query whether it's correct or not uh based on the relevant document and then the third one query rewriter okay so here what I'm doing uh let's let me create a couple of more lines over here and here the next

thing is going to be query grid okay so I written the query grader also just a second let me give you that code as well and then one by one step by step I will explain you each and everything so here is my uh query grader guys this one now

let's try to create a query grader and query sorry document grader based on the given query okay now here is what here is my uh basically class this is my penic object okay and what I'm doing I'm passing this penic object to my model

itself okay So based on the uh given query basically given query in this document it's going to be great whether this uh whether this thing is relevant or not means whatever question I'm passing it is relevant to this document or not okay now here if I want to test

it so this is my simple code for testing the document sorry testing this grader okay this document grader whether my grader is whether my document is correct or not means whether my uh this document basically it is it is matching with the question or not not okay simply I'm

checking now what I'm doing here so here I'm executing it and yes we are able to create a pending class now here is what here is my LLM okay and this complete pipeline right retrieval grader and here I'm just checking whether it is working

fine or not so this is my query and here is my relevant document and once I will invoke this chain simply it will be checking whether this uh document whatever is coming it is relevant to this query or not okay so here I'm executing it and see here uh what I be

LangGraph:13 Corrective RAG for Real Time AI Application #llm #genai #aiagents #ai #langchain #genai

Welcome to my YouTube channel, where I empower minds with data! 

My name is Sunny Savita, and I'm a data scientist and AI engineer with almost four years of expertise across many domains. On my YouTube channel, I provide high-quality, free videos about everything data-related. My goal is to make complex data science concepts more understandable.

Please subscribe and support this channel.  I commit to create more interesting contents as we move forward.


Sunny Savita

RAG에 초점을 맞춘 엔드투엔드 LangChain 과정의 13번째 동영상을 소개합니다.

수정 RAG, 에이전트 RAG, 적응형 RAG 및 셀프 RAG를 다루는 예정된 동영상 개요입니다.

나머지 주제를 강조하는 코스 강의 계획서 프레젠테이션: RAG 및 멀티 에이전트 시스템

LangChain과 AI 애플리케이션 구축의 맥락에서 RAG에 대한 심층적인 이해에 중점을 둡니다.

관련 콘텐츠에 쉽게 액세스할 수 있도록 YouTube 채널에 정리된 재생 목록에 대한 정보입니다.

RAG(검색 증강 생성)에 대한 간략한 개요와 데이터 주입, 검색, 생성의 세 단계에 대해 알아보세요.

표준 RAG에서 제공하는 오답을 수정하는 데 중점을 둔 수정 RAG를 소개합니다.

유연성과 루핑 기능을 강조하면서 수정 RAG를 구현하기 위해 LangChain 그래프를 사용하는 방법에 대해 설명합니다.

초기 RAG가 실패할 경우 수정 RAG가 인터넷에서 데이터를 가져와서 정보를 수정하는 방법에 대한 설명입니다.

LangChain 그래프를 사용하여 수정 RAG 아키텍처에 대한 자세한 설명과 워크플로우를 보여줍니다.

코드 구조 및 위치 개요(GitHub 리포지토리).

모델을 로드하고 기능을 확인하기 위한 코드(Google의 JAX LLM)입니다.

리트리버 생성, 데이터 주입, 벡터 데이터베이스를 사용한 유사도 검색을 수행하는 코드입니다.

RAG 체인을 생성하고 기능을 테스트하기 위한 코드입니다.

웹 로더를 사용하여 블로그 글 URL에서 데이터를 가져오고 데이터를 청크 처리하는 방법에 대한 설명입니다.

LangChain의 상태 그래프를 사용하여 상태 및 워크플로우를 정의하여 수정 RAG를 생성하는 코드입니다.

리트리버, 그레이더, 생성기, 쿼리 변환기 및 웹 검색을 위한 함수 생성.

주어진 쿼리에 대한 문서의 관련성을 확인하기 위해 쿼리 그레이더를 만드는 코드입니다.

문서 관련성을 기반으로 쿼리를 변환하는 쿼리 재작성기를 만들기 위한 코드입니다.

수정 RAG 아키텍처 및 워크플로 검토.

실행 프로세스에 대한 단계별 설명으로, 시스템이 관련 문서와 관련 없는 문서를 처리하는 방법을 보여줍니다.

아키텍처 및 데이터 소스에 대한 잠재적인 수정 사항에 대해 논의합니다.

수정 RAG 파이프라인의 최종 코드 실행 및 데모.

관련성 있는 쿼리와 관련성 없는 쿼리의 예와 함께 코드를 최종 실행합니다.

시청자를 위한 결론 및 콜투액션.

LangGraph:13 Corrective RAG for Real Time AI Application #llm #genai #aiagents #ai #langchain #genai

LangGraph:13 Corrective RAG for Real Time AI Application #llm #genai #aiagents #ai #langchain #genai

스크립트