Hello, this is the happy squirrel. In this video, I will introduce Chainlit and Streamlit, which allow us to create an interface, or web screen, connected to Python code. We will use these tools to create the interface for the Sherlockian AI that we developed in previous videos. This video continues from the one where we made the Sherlockian AI using RAG. Chainlit is an open-source Python framework designed to facilitate the rapid development of interactive AI applications. By using Chainlit, we can create applications that utilize the Python code we created in earlier videos. Chainlit is compatible with libraries for AI, such as OpenAI and Rankchain, and provides a chat-style interface, allowing us to create applications in the form of ChatGPT. Now, let’s create an interface for the Sherlockian AI using Chainlit. The first task is to perform indexing for the Sherlockian AI. Unlike the previous video, in this one, we will save the Vector Store to a local storage and load it from there.

First, we will use the four stages of indexing: Load, Split, Embedding, and Vector Store to import external data, convert it into vectors, and store it in the Vector Store. It is important to remember the Embedding Function during this process because later, when loading the Vector Store from another file, information about the Embedding Function will be necessary. In the Retriever stage, the Retriever converts the question into a vector and performs a similarity search with the data in the Vector Store. However, if the method of vectorizing the question differs from the method used for the vectors in the Vector Store, we will not be able to use an appropriate similarity measure. Therefore, it is essential to remember which Embedding Function was used during the indexing stage. Now, we can use the Save Local function to save the vector storage to the user's computer. Once saved, you will see a folder named Sherlock Holmes created in the folder where the indexing files are located. If you want to save it to a different path, simply write the file path in front. Now, let's proceed to create the Chainlit Interface. The Chainlit Library requires a Python version of 3.8 or higher. You can install it by entering the command pip install chainlit in the terminal.

The second task is to retrieve the code we created in the previous video. First, we will bring in the LLM model and the prompt template. Secondly, we will specify the Embedding Function that we designated during indexing. Since we used OpenAI Embeddings without specifying parameters, you can simply specify the same Embedding Function. Next, we will use the logged local function from FIAS to load the saved Vector repository. At the beginning, we will save the path to the Vector repository. I have saved the Vector repository in the same folder as this code, so I only wrote the title. The next parameter is for the Embedding Function. The last parameter asks whether to allow actions that may be risky when transforming data stored as bytes back to its original form. This parameter exists because allowing risky actions when using untrusted external data can make the system vulnerable. However, since I am currently using the data I indexed, I set this parameter to True. Then, we will create a Retriever using the as_Retriever from the Vector repository.

A RAG Chain has been created using these elements. This concludes the brief explanation of the components used in the previous video. Now, let's get started with Chainless. In this code, Chainless is imported as CL. The first thing to do is create a start function that will be executed when the application runs. To specify the start function, you need to decorate the function with @CL.OnChatStart. This way, the specified function will run when the application starts. The ASYNC in front of the function indicates that it is an asynchronous function. Making a function asynchronous prevents the program from freezing while the user is performing tasks, allowing it to serve multiple users simultaneously. This is an important factor when using the service directly, so it is recommended to use asynchronous functions in Chainless. The start function can perform various tasks, such as displaying a starting question like What brings you here? or checking the user's profile. For now, let's simply display the greeting message from Sherlockian AI.

The message from Chainlit is a class that contains information about the message, such as its content and author, which will be used in the chat. The content will include the message itself. First, we created a message object by inserting the greeting from Sherlockian AI. To send this message object, we can use the Send method. We utilized this method to send a greeting message when the application starts, allowing the user to see the greeting. The Await keyword at the very beginning is used to call an asynchronous function defined by the preceding Async keyword. It is employed to use the asynchronous function Send. By defining a start function this way, you will see Sherlockian AI's greeting appear in the chat window first when Chainlit is executed. Now, let's create a function that reacts to messages when they arrive for the user. By specifying the CLOnMessage decorator above the function, that function will be called every time a message arrives. The parameter of the function that comes under OnMessage must be of the CLMessage type.

The parameter will receive messages from the user. Therefore, I specified the type hint for the parameter as CL.Message. Now, we need to insert the content of the message we created earlier into the Chain to get the response from the LLM. After that, we simply need to select the content of the response and send it as a CLMessage, just like before. Now, let's run the Chainlit program we created earlier. Please open the terminal and enter Chainlit Run. Next to that, enter the path of the file you want to execute. By doing this, the Chainlit Application will run on Localhost 8000, which is the server of our local computer. However, when you are directly creating and testing code, it is advisable to enter W next to the area to enable Watch Mode. Watch Mode provides a feature that automatically detects changes in the code and restarts the program. It can be quite cumbersome to have to restart Chainlit every time you want to change the code while testing the program, right? However, by using W to enable Watch Mode, the application will automatically restart when it detects that the code file has been changed and saved.

You can use this mode to conduct smooth testing. Now, I would like to introduce the second library that allows us to create an interface: Streamlit. Streamlit is an open-source Python framework that helps you quickly build and unify web applications for data visualization. Its ease of use is a significant advantage, and it is currently being utilized not only for data visualization but also for creating LLM applications. Now, let's create an interface for Solukhan AI using Streamlit. First, we need to install the Streamlit library. Then, we will import the elements necessary to create Solukhan AI, such as models and chains, just like we did with the Chainlit interface. Now it's time to write the functions for the interface. Unlike Chainlit, Streamlit does not specify separate functions for the start and for responding to messages. Therefore, we will create and use only the main function in this code. First, let's create the interface that will appear when we start.

Here, st refers to the Streamlit library. First, we create a title with Sherlockian AI using st.title. This will make the title Sherlockian AI appear in bold on the screen. st.write is a method that displays text in the interface. You just need to insert the text you want to display as a parameter. I added a greeting for Sherlockian AI. st.text_input creates a text input box and is a function that reacts to the current input in that box. The first parameter of the function is the label that appears right above the text input. I placed a message here asking users to enter a question. Finally, st.button is a function that creates a button. The first parameter is where the name that will appear on the button goes. I created a button labeled Send. When this button is clicked, the code under if st.button will execute. I will explain this in more detail shortly. According to this code, the initial interface of Sherlock Holmes AI will be created.

First, a title called Sherlockian AI has been created using st.title. Then, a greeting message has been written in st.write, and below that, there is a label asking for a question and a text input box. Finally, a Send button has been created. The initial interface has been built with just four lines of code. Now you can understand why Streamlit is said to be easy to use. Next, we will write the code that responds when the user enters a message in the text box and clicks the Send button. As mentioned earlier, the text_input function returns the content entered in the text box. Therefore, user_input holds the string from the text box. When the Send button is clicked, the corresponding if statement will return True. This will trigger the code within the if statement. Upon clicking the Send button, we first provide user_input to the chain created in the previous video to obtain the LLM's response, and then we display the content of that response in the interface using st.write. Finally, we add the code to execute the main function, and it will be complete. The conditional statement in the code ensures that the main function is called only when the Python script is executed directly.

We will be executing the script directly using St.write, so we have included that conditional statement. Now, when we run St.write and input a question in the text entry box and click the Send button, the answer from the LLM will be displayed on the screen shortly after. However, what should we do if we want to ask another question? Do we need to restart Streamlit? Not at all. Due to the nature of the Streamlit application, the entire code is re-executed every time the user interacts, that is, every time the Send button is clicked. Therefore, by entering a different question and clicking the Send button, you can obtain another answer. Of course, there are also ways to maintain the state even after interactions, but this is not a necessary feature for this Sherlockian AI, so I will explain it in a later video. Now, shall we run Streamlit? The method to run Streamlit is similar to running Chainlit; you just need to type Streamlit run followed by the file path name. Then, like Chainlit, the application will be executed through a local server.

The way to run Chainlit differs in that Streamlit does not require a separate watch mode. When you run Streamlit and make changes to the code, it detects changes without needing a specific mode. However, unlike Chainlit's watch mode, which immediately reruns the application upon detecting changes, Streamlit displays a notification at the top right of the screen saying Source File Changed. If you want to reflect the changes, you can click the Rerun button next to the notification to restart the application. Which library should we use between Chainlit and Streamlit? Chainlit is a library specialized for LLMs, allowing for more precise adjustments suitable for LLM applications, such as gaining insights into the reasoning processes of LLMs. However, it is more challenging to use compared to Streamlit. Streamlit is easier and originally designed as a data-related library, giving it an advantage in data visualization. However, it is not a library specifically created for LLMs like Chainlit.

However, according to a recent interview with the COO of Snowflake, the owner of Streamlit, Streamlit also offers several features that can efficiently create LLM spots and is planning to develop more. It could be a good option in this regard. Compare the advantages of both and choose the better one based on your situation. The Chainlit section of this video was created based on the three chapters of Samurai Haruka's perfect guide to LangChain. In the future, I plan to create coding videos that will progress towards building interfaces using Chainlit and Streamlit. Thank you for watching. Always stay happy.

chainlit&streamlit으로 llm 인터페이스 만들기

chainlit&streamlit으로 llm 인터페이스 만들기

Scripts