[Music] [Music] What's up guys? This is Ronnie, welcome back to our channel, Total Technology Zone. This is tutorial 34 and today's topic will be a little bit advanced and very important topic. So in this tutorial, I'll be going to show you guys how to

use multiple documents and with the help of multiple documents, how we can create an agent, and with that agent, we'll be going to develop a RAG. And in the RAG, we'll be going to use Llama index query engine tool. Okay? So there are four things: multiple documents, with the help of

multiple documents, we'll be going to create an agent, and that agent will use the query engine tool, right? And all together everything will be going to be uh used for a RAG. Okay? So for the sake of simplicity, I have created uh two documents, one is AWS

and one is DevOps. So let's say if the user is going to ask anything related to AWS, this document will be used. And if the user is going to ask anything related to DevOps, this document will be used, right? Right, so let's start working on the

tutorial. But before I start the tutorial, uh the source code link is actually available here. I don't like to talk much on the theoretical concept. All of you know, so let's start uh directly the coding part. Okay? So first thing first, we'll be going to Let's maximize this

thing. Okay? And first thing first, from Llama index. Okay, Llama index core, uh it will be import SimpleDirectoryReader and VectorStoreIndex. Okay? So we'll be going to uh import that. Then from Llama index. Okay, Llama index.llms, we'll be going to import OpenAI. Okay?

so open a so this two things uh important so first thing as I said I have two documents so let's create two loaders first uh AWS loader okay in the AWS loader what we will do we'll be going to load the document from a this so it will be

simple directory loader input uh input maybe input files right and this input file will be going to be like copy path in this so like this okay done and then uh AWS docs so AWS docs okay equals to to AWS loader. load data right done now we'll

be going to create an AWS Vector store index so let's say AWS index index will be um Vector store index okay let's say prom documents and inside that I'll be just going to use AWS docs right ready and then we'll be going to create an a W quy engine

AWS uh AWS query engine QE okay this will be let's say uh AWS index dot s uh query engine and here probably I can write [Music] Sim what is the similarity _ topk something will be there uh I didn't find that similar

it underscore top underscore K = to 4 okay everything is ready so let's execute this thing and see whether I'm able to get something or not okay so this is actually from the previous code so let me clear this thing yeah let's execute one more time yeah

it's working let's see okay done so now what will happen let's see okay okay so right so uh AWS simple directory reader okay so now what we'll be going to do we'll be going to write debox loader deop stocks deop stocks deop Index right de

Oops index and the Oops query engine okay the Oops okay so first thing I'll be going to change it here so it will be the of St okay the of Stf right so I'll be going to use it will be devop loader so Dev op op loader so devop

loader is created here and that deop Lo will be used to create the deop stock NOW deox index will be used to Vector store index the of Dos da op dos okay da op dos then this is AWS index I'll be going to D OB

index da OBS index okay that's it now let's execute and see if no error it means everything is working fine okay okay no error means everything is fine now with the help of this two document I'll be going to to import couple of more things now we be going to

develop a tool actually so query engine tool so from Lama index so Lama index.core okay uh do agent so it should be function agent import okay functional function calling agent right so this function calling it will be function calling agent worker or something make a function calling agent worker right so

one more module is uh I need to import let me check if I forget so basically another one will be U query engine tool and Tool metad data right so from okay so from Lama index.core do tools okay import squ engine tool and another thing

is actually tool metadata that's it right so first thing first let's uh create something called query engine tool Q okay and there I'll be going to sarey engine tool right okay so now uh basically um basically this is actually going to be a list okay this

is actually going to be at least so what I'll do I'll just going to make it like this okay right this this entire thing will be in a list and the square engine tool will be going to be used as a uh we'll be going to use an object of the

square engine tool so first uh thing will be actually query engine so query engine so this query engine will be uh let's see uh let me tell you what is what is going to be there so the square engine will be um I'm just thinking hold on so it will be uh AWS Q

AWS right this is done then there will be a comma okay and there should be a metadata and metadata equals to what so metadata equals to tool metadata for the tool metadata so inside the tool metadata again I'll be going to use something called let's say

name okay so name probably I can just write any name AWS docs comma and description I can write it let me think what I can write okay uh so okay provides uh information and on [Music] anything a related questions okay okay then so let's let's refresh

this thing so query engine tool is actually completed here this is the first parameter query engine tool then this is the second parameter so what I'll do I'll be just going to refresh this thing okay and here uh okay so this is tool metadata and this is finished

here right so what I do I'll just going to uh put this bracket here so this Square engine tool is going to be here and there must be a comma okay done so this is done this is the first parameter then there will be a second parameter second

tool actually sorry this is at least so this is the thing okay okay so just hold on okay so what you'll do so this is going to be deop uh QE so da Ops query engine tool tool metadata equal to daop stock this is just a name remember

okay my keyboard is actually very slow just hold on I'm using actually a Bluetooth keyboard so I'm getting a very very slow response from my thing I don't know uh D Ops Doss okay so first tool is using awv and second tool is actually

deop all done so let me um do this thing and see whether I'm getting any error or not okay no error so what we will do now we'll be going to actually uh create let's say next thing is actually uh we be going to create uh

this is square engine tool now we're going to here agent worker okay so agent worker will be function agent worker uh dot some tools so first thing will be our tool so the tool I'll be just going to mention you query engine tool LM is actually

LLM can uh okay I didn't put create an LLM so LLM equals to open AI LM okay open AI and there going to put model G then okay so now this equals to then hello parallel C is going to be false okay then all done right now we'll

going to create agent so it's equals to agent worker do F agent okay all done now let's write response chat okay let's ask uh provide uh uh write briefly about what is just happening I don't know sorry guys sorry right let me copy this SC first

right this okay about okay and we can just print response okay that's it so let's see it should get the answer from AWS Square in okay see it is using this one as an argument and it is getting the information okay fine right okay good now let's see um let's go it

quickly [Music] about devops okay let's ask okay so now you see here see the calling function it is calling the AWS stocks this is AWS stocks right and here when I ask DevOps it is actually calling the DevOps stocks okay so now let's say let's ask some tricky

question let me write okay so let's ask probably the features of Dev let's see whether it is able to get it from not hopefully it will do it because I'm using GPT 4 so in this way guys you can use uh multi-document

RAG okay nice nice nice okay so let's ask something from AWS so let's see okay benefits of using AWS what what are the the it are using a let's see it should use AWS docs again okay it is querying AWS cool my God this is this is very very very

good guys okay so I'm actually impressed uh by the thing okay hopefully you guys are also excited to try something out with this thing right so see uh the code is actually a little bit complicated but uh if you practice it won't be that much of complicated you have to only

understand and try to watch the video see if you watch the video just in a flat mode it will not work my recommendation is that you should watch the video and after watching the video you should try out this code by your own maybe you can create some other use cases

as well see these things are very very small tutorials or basically I can create the small small use cases and as much as possible industry oriented use cases I'm trying to do it but see when you are going to do something in the company there the data magnitude will be higher

and also you will get some very very uh highly customized uh like use cases so in that time you should not get uh like any any trivial so that is why I'm just trying to create this tutorial more or less like use case specific okay so see

I as I told you like I'll be going to do it with the basic thing and gradually I'll improve the complexity and gradually I'll try to include the project so probably this is a one of the very important project and also in the last tutorial we did another project so

see this two three projects you can actually uh also add any add any add inside your CD or maybe in your GitHub repository and after that all together you can put it your LinkedIn profile or maybe in your CD again right so if you have some other use case you can try out

similarly right okay so last but not least before I conclude I'd like to say a couple of things guys please try to um try to check out to my YouTube channel or other playlists as well if you go to the playlist section you will find there

is one playlist on LangChain it has more than 130 videos it's a complete playlist I created and the work is going on and hopefully by mid of uh mid of the uh mid of December more videos will be coming on the LangChain I I have started

creating this Llama Index series and if you see here this is the 34th tutorial, right? So more videos will be coming. Next thing I also started creating videos on uh computer vision using OpenCV, the latest edition which is version five. So I already posted 10 videos, try to watch

this thing as well uh because this is also going to be a very, very, very popular uh nowadays because I'll be going to do on computer vision and deep learning together and for that computer vision knowledge is important. And to master the computer vision, you should know a

image processing library from Python, OpenCV. So all these 10, 10 tutorials are actually uh more or less talks about those basic like operations and key processing and processing techniques from OpenCV. And last but not least, I created another very popular playlist on Hybrid AGL. Try

to watch those playlists as well. Okay, so please try to subscribe to my channel, hit the like button, share the videos with your family and friends. Ask them to also uh watch the other videos as well. I need your support basically, guys. If I don't get support and engagement, like my

channel will not grow and if the channel will not grow, it will be very difficult to maintain this channel like so. I just need a little help from all of you. Please click the subscribe button, try to share my videos and we are uh engaging you, like I need some engagement from all of

you because the moment you guys have started subscribing, commenting on my videos, YouTube will also going to promote the videos to a large number of audience. And also so my ultimate objective is to reach a large number of audience. So guys, please, please help me. I need your support to

Okay, so enough of talking and marketing. I did sorry for that, but again it's just a thing, right? Okay, so that's it. I'm just going to conclude the video over here. We'll see you in the next video. Till then, take care. Goodbye.

multi document agent based rag using LlamaIndex query engine tool

Total Technology Zonne

라마 인덱스를 사용한 RAG 소개 및 개요

RAG(검색 증강 생성) 시스템 생성에 대한 튜토리얼을 소개합니다.

네 가지 주요 구성 요소인 다중 문서, 상담원 만들기, 쿼리 엔진 도구 및 RAG에 대한 설명입니다.

데모를 위해 두 개의 샘플 문서(AWS 및 DevOps)를 사용했습니다.

튜토리얼의 코딩 부분을 시작합니다.

환경 설정 및 필요한 라이브러리 가져오기

라마 인덱스에서 필요한 모듈 가져오기.

AWS 및 DevOps 문서용 로더 만들기.

AWS 및 DevOps 문서용 VectorStoreIndex 생성.

AWS 및 DevOps 인덱스용 쿼리 엔진 만들기.

쿼리 엔진 도구 및 에이전트 개발

상담원 생성을 위한 추가 모듈 가져오기.

메타데이터를 사용하여 AWS 및 DevOps용 쿼리 엔진 도구 정의하기.

쿼리 엔진 도구와 OpenAI LLM을 사용하여 에이전트 워커 만들기.

AWS 및 DevOps와 관련된 다양한 쿼리로 에이전트를 테스트합니다.

쿼리에 따라 올바른 문서를 선택하는 시스템의 기능을 시연합니다.

보다 복잡하고 미묘한 쿼리로 테스트합니다.

마무리 발언과 함께 강령 실천을 독려합니다.

보다 복잡한 사용 사례를 만들기 위한 제안 사항입니다.

포트폴리오에 프로젝트를 추가하도록 권장합니다.

LangChain, OpenCV 및 하이브리드 AGL에서 다른 재생목록을 홍보합니다.

행동 유도: 동영상 구독, 좋아요, 공유.

multi document agent based rag using LlamaIndex query engine tool

스크립트