hi friends. Graph rack from Microsoft is quite advanced, and I think it's the best rag architecture so far. Let's say we have a book of 100 pages. We build a rack system. Let's say we divide each page into one chunk, so we have 100 chunks, and we create 100 embeddings. We store them in the vector store. Now, when we ask a question, we typically extract the top five or maybe even 10 chunks which are similar to our query and provide only those few chunks as a context to the LLM. So, the LLM response is going to be based only on the retrieved documents, which are very small compared to the actual book, right? Now, this works fine when we have very specific questions, but let's say we are asking questions like what are the main ideas discussed in the book, or what are the key milestones in the author's life, or provide me a high-level summary of the project the book is about. Let's say the project. Right to answer questions like these, the LLM needs to have full information about our documents or the

book right so in such cases, the traditional rack with few retried documents will not work, and this is where graph rag shines. Okay, so we have traditional rag, which is based on the vector store, and then the frameworks like llama index, neo4j, they have already incorporated this knowledge graphs ideas into rag systems. So, in addition to the vector store, we have entities and relationships in the form of Knowledge Graph. But what Microsoft has done is they took it several steps further. In addition to Vector store and traditional Knowledge Graph, they have introduced a few concepts like communities, hierarchical communities, community reports. These claims are covariant. So, these four additional concepts in my view makes it the best rack system so far. Okay, um, all right, let's see the workflow here. We have, yeah, so we start with a bunch of documents, we chunk them, we create the embeddings. This is what we do for a traditional rack, right? So, we take these chunk documents, which we call these text units, and in step two, we extract these entities and relationships. Okay, and then we do some further

processing of entity resolution if two entities are the same. Uh, uh, so we dup the entities, so those sorts of things we do, right? So, step two is also already known, right? So, Step One is the traditional rag, step two is building this knowledge graph, and then Microsoft introduced this community detection. So, using these entities, we develop the communities, okay? And these communities are hierarchical communities, and then for each community, we also create these community summaries, okay? Now, uh, let's say we are talking, uh, uh, our documents are about geography, right? So, the communities can be each country, can be one community, and within that community, we can have the sub-communities, which can be cities, right? So, we are summarizing or creating reports of these communities, sub-communities, uh, so on and so forth, right? So, we are condensing the information at different levels of hierarchy, okay? And we combine all this information, uh, so this fifth step, it's, uh, mostly the mapping step, uh, between uh, the documents, the chunk, uh, the text chunks to the uh, uh, entities and relationships, so on, so

forth okay okay all right all right so let's see uh how we can uh run it okay so first here uh I have created a folder called this rack test and then uh we install the library. It's a simple pip install and then create an input folder and place you your documents. So currently it supports only text documents so you can place all your text documents okay if you have PDFs Etc first convert them to text and place uh your documents right. So to be begin with we simply have input folder and the text file okay and then first we initialize uh this index okay so run this python command. What it does is it creates this output folder. It creates this prompts folder uh with these four files and it also creates these two files environmental file as well as the settings file. Now we go into these prompts and the output generated files in another video because uh that uh uh uh it's quite involved but as you can see from here uh these prompts uh not this one claims extraction yeah. So these prompts those are very very detailed.

right so they uh extract lots of information. So go through these prompts, and the only change uh we need to make is in the environmental file. We need to provide our OpenAI key, okay uh don't worry uh here it's called this graph rag API key, but here you can provide your OpenAI key. Uh maybe in another video we will see uh how to use open-source LLMs as well as embedding models uh to uh uh to run the whole system again. All right now the important file is this settings uh file. Uh again you don't have to touch anything. Uh I have been doing some experimentation, but by default here you will see an OpenAI model. Uh again as I mentioned we can make use of uh open-source models, so we need to make some changes within this file like API basis uh uh and the model name uh for the LLMs, and similarly uh for the embeddings also. Here we have the model name and the API basis, okay. So to use open-source models we need to make a couple of changes. We can make use

of AMA Etc. Uh, we'll discuss that, uh, maybe in a separate video. All right, uh, let me go here. Yeah, so we initialize the index that will create the prompts and these two files and along with this output folder. Okay, and then we run this command which actually processes our input data to create all the required files, right? The vector store embedding, so and so forth. So this is how it looks like. Yeah, all right, so here we are. Yeah, so here I'm running, uh, the command, so creating the index. Uh, so we have one document, uh, that is broken into 230 chunks with each chunk having about these, uh, 300 characters. Uh, and then here we are extracting the entities and summarizing the entities, building a graph based on entities, and then these are the final, uh, version of entities. Uh, these entities we convert them into graph SL, these nodes, and we detect these communities, and we map the text to these entities because these entities are based off of, uh, the text, right? And then, uh, again doing some mapping between entities and relationships. Uh

and here we create. Uh, this is another mapping step where we map the text. Uh, documents are the chunks. Uh, into their documents, right? We can have multiple documents, right? So, as you can see, it's quite involved and there are a number of steps. And if you see this, all workflow completed successfully. That means, uh, it ran, uh, fine. Now, even for, uh, documents with moderate size, uh, it is taking about, uh, 20 minutes, over 20 minutes with OpenAI and making multiple parallel calls, etc., right? So, the two cons, uh, with GraphRAG are one, it's quite expensive. Now, indexing or processing the documents itself cost me about $6, and also the process is extremely slow. Okay, so those are the current, uh, uh, uh, cons of, uh, this method right now. Once this process is complete, we will see these timestamp-based, uh, folders in the output, uh, folder, and within it, you will see, uh, these artifacts and reports. Okay, so the reports contain the logs, but, uh, the more important one is these artifacts. As you can see, a number of files have been

uh generated so these are generated at the beginning, but the final ones are the important ones. So, there will be six uh these create final uh packet files. Okay, so here we have um uh the communities. Those are based on uh these graphs and Community reports. So, the summarization, as I mentioned, for example, a city-level summarization, then country-level summarization, then continent-level summarization type of thing. So, that depending on the user question, uh we can extract the appropriate Community-level information. And then this is just a mapping one. The final documents we have, the entities, the relationships, the graphs, and the text is the chunks in uh traditional graph sense, right? So, these six documents are used when the user asks a query. Okay, all right, so we build the rack system and then we ask questions, right? So, to ask, there are two modes. So, as we discussed, for these types of questions, we require this Global or comprehensive View, and we can also ask specific questions uh for which we don't need uh uh we need a different set of uh information to be provided, right, as

context so to ask a question, we simply say graph rack query. We specify the method, let's say it's Global, and here we are asking what are the top themes discussed in this story. Okay, so this is how the output looks like. Now, I ran all this before. I could not do live because it's taking quite a while, so I'll show you. So here is our query. So we are asking, so it's a global method, what are the top themes in this story. Okay, all right, so here you will see success. So the global search response. So this book is this Christmas Carlos written by diens. Now, as you can see, the top themes are like Community Building, celebration of Christmas, compassion and generosity, moral complexity, and so forth. So if this were a traditional rack system, out of let's say this 100-page book, we would have extracted maybe top five pages, right, or top five chunks. Now, those chunks might have talked about let's say these three or four.

themes we would have missed the remaining, uh, themes, right. So, to answer these types of questions, we need to provide the LLM a high-level summarized information because we cannot provide the full book as the context, right. If you were to do, then we wouldn't need these graph racks, knowledge graphs, etc. We wouldn't need anything, right. We can provide the full books, uh, as the context, right. So, that's why using this graph rack, we are summarizing the information at different hierarchical levels through these, uh, communities, right. And we are using those communities to answer the global questions like these, okay. Now, I built another system, uh, using, uh, this Paul Graham, uh, uh, stories, uh, sorry, um, uh, the essays. So, here I have asked a question, summarize Paul's work. Now, this is not based on all his 220 plus, uh, uh, essays, uh, only based on a couple of essays. And here, as you can see, uh, it divided, uh, into, uh, so this programming and entrepreneurship essays, co-founding y.

combinator writing projects uh before the college uh and then the books. Okay, so it summarizes uh his work in these four different themes. Okay, now similarly, let's see how we can ask a local question. So it's exactly the same uh the method is simply local and here we are asking so who is uh Scorch and what is his uh main relationships, right? So this is a very pointed specific question. A traditional rag system would also do probably because uh we are asking who is this person and what are his main relationships, right? So when we extract those four or five chunks, they probably contain all the information to sufficiently ask uh answer this question. So here uh based on my second system here, I'm say simply asking where did Paul study and where is pointed question. So local search uh is successful and it provided uh me with this uh response. Okay um all right, it's quite involved as I mentioned uh so maybe in the follow-up uh I'll do a couple of follow-up videos uh one using uh how to build the whole

system uh using uh open-source llms. As I mentioned, it's quite uh costly. A simple book uh cost me about some $6 just for building the index. Okay, so uh we'll test with uh open-source llms, and we will also unpack uh what are, what is in all these files, and how they are related to each other. And we'll also visualize uh these graph uh, uh the nodes, entities, uh how they related to texts, uh the community, sub-communities, uh Etc. All right uh, that's all for this video. Thank you very much.

Microsoft GraphRAG: The Best RAG + KG Architecture

Learn Data Science, ML, AI and Cloud Computing through step-by-step hands-on tutorials.

Subjects covered include,
Data Science
Machine Learning
Deep Learning
Predictive Analytics
Artificial Intelligence
Generative AI
Large Language Models
Natural Language Processing
Computer Vision
Transformers
Embeddings
Vector Databases
GraphML
Data Visualisation
Business Intelligence 
Monte Carlo Simulations
Discrete Event Simulations
Genetic Algorithms
Optimisation
Python
AWS, Azure, GCP

Pls do not hesitate to contact me if you have any questions or feedback. Pls share the channel with anyone/group who might be interested. Thank you :)

Sridhar Kumar Kannam
https://www.linkedin.com/in/sridharkumarkannam/
