Storm: AI-Powered Wikipedia Generation Tool

Storm, an AI project from Stanford, creates Wikipedia articles on various topics, using referenced info from web pages.

### Stanford's Storm AI: A Powerful Wikipedia Generation Tool
* 🤖  Stanford's Storm AI is a cutting-edge tool that can **generate entire Wikipedia articles** on any given topic, utilizing AI research agents to **compile information from various sources**.
* 🌐  The tool is open source and can be run locally, with the exception of the web search function which currently relies on the **OpenAI API**.
* 🔧  The installation process involves **cloning the GitHub repository**, **setting up a Python environment**, and **creating API keys for OpenAI and u.com for web search**.
* 📚  The generated articles are well-structured, covering topics such as **taxonomy, evolution, physical characteristics, and behavior**, with references to the **original sources**.
* 🖥️  A user-friendly UI is available through Streamlit, providing a **simpler interface** for conducting research and viewing the generated articles.
* 🔍  While the tool currently supports **OpenAI** and **u.com** for web search, there are plans to add support for other search engines like **vlm** and **mistol** in the future.
* 💬  The creator of the tutorial offers to do a **full local setup guide** if there is interest, and encourages viewers to **like and subscribe for more content**.

we have a new project out of Stanford, which is the best AI researcher that I have ever used. Simply give it a topic, and it is going to create an entire Wikipedia about that topic. You can run the whole thing locally, and it works really well. The best part: it will actually reference each of the websites that it got the information from in the Wikipedia itself. That is what you're seeing here. So, I asked it to create me a Wiki based on artificial intelligence, and this is it. It gave me business and industry, healthcare, education, security, basically an entire team of AI agents researched this. Then, for each of the facts that it output, it actually referenced the web page that it got it from. And today, I'm going to show you how to install it, get it all set up, and get it running. And you can run everything locally if you want. The only thing you can't run locally is the web search. So, you can see it's open source. It is running locally on my computer, although we are using the OpenAI API right now.

but you don't have to now. Before I show you how to set this up locally, you can actually test it without setting it up at all. They have a demo page at storm.genie.stanford.edu. And you can see here, they already have pages that are fully researched right here, very very impressive. And one thing that looks like it's coming is this little toggle human AI collaboration mode available soon, which also looks very cool. So now let me show you how to install it. All right, so here's the GitHub page. It already has nearly 62,000 Stars. Hasn't been out that long. Storm stands for Synthesis of Topic Outlines through Retrieval and Multiperspective Question Asking. Now, I did run into one bug and I actually had to change the code to get it working locally, but I'll show you how to fix it. All right, so the first thing you're going to do is open up VSS code. CD to whatever directory that you want this stored at. So I usually like to put

everything on my desktop. Just when I'm first testing it out, so that's what I'm going to do. CD desktop. Next, we're going to come back to the storm GitHub page. We're going to click this little green code button, and we're going to copy this GitHub URL right there. Then we come back to our VS code, and we type git clone. Then we paste in the URL that we just copied, hit enter. That's going to clone that repository to our desktop. Once we've done that, we're going to CD into storm. Next, we're going to use our favorite Python environment management application conda. So conda create DN storm python equals 3.11. Now, I already have an environment called storm, so it's going to ask me if I want to overwrite it, and yes, I do, but you won't see that. Okay, next, I'm going to proceed. Then we grab this command right here, conda activate storm. We're going to paste it in, and then we activate that environment, and we can tell it's active because it

says it right there. Now, from there, we're going to install everything in requirements.txt. So, pip install dasr requirements.txt and hit enter. All right, so that's done. Now, what we're going to do is we're going to go up to file and say open folder, go to the desktop, and we're going to open up Storm. Now, it does reset our terminal, so let's open that back up. And the first thing we're going to do is conda activate Storm, hit enter. So, now we have it active again. And over here, we're going to create a file called secrets.toml, and I'll show you how to do that. So, you're going to right-click new file and call it secrets.toml, and then hit enter. Now, back on the GitHub repo page, we're just going to copy all of this, which is under the getting started header. Copy that code, come over here, and paste it in. Now, the first time going through this, I'm going to show you how to do it with OpenAI, but you can do it through the OpenAI

API, but you can also do it through Azure if you want, but we're not going to be doing it through Azure. So I'm simply going to delete that, delete this line of code as well. So the OpenAI API type is going to be OpenAI. So the first thing we're going to need is an OpenAI API key. So if you don't already have an OpenAI account, go ahead and create one. And then you go to platform.open.com. AI keys. We're going to click this button, create new key. And I'm going to call it Storm YT because I will revoke this key before publishing this video. Grab the key. We're going to switch back to VS code. And we're going to paste it in on line two. And then we can hit save. The next thing we need is a ud.com search API key. And U.com does cost money, but they give you a free trial. So you don't actually have to pay anything if you just want to try this out. So you go to api.u.com dashboard.

It'll ask you to sign in. Go ahead and sign in. We're going to click this create API key button right there. Here under web search, we're going to click generate. And then here is our API key, which I will copy. So, I couldn't actually find a way to revoke API keys from u.com, so I'm not going to show that one. So, we're going to paste in the u.com API key right there and then hit save. And then you're pretty much done. So, there are two ways to get this working. One, you can do everything command line, which is nice and it outlines all of the different options that you have right on this page. But to make it a little bit easier, they created a very simple UI that we can use, and that's my preferred way. It just makes everything much simpler. So, to get that working, you're going to look for this folder called front end. And then within that, there's a streamlet folder. You're going to right-click on secrets. tommel and then

copy it CU we want it to also be in the streamlit folder, and then you're going to right-click on this do streamlit, and click paste, and that will make it available to the front end. Okay, the next thing you're going to do is CD into the demo light directory, then you're going to run streamlit so streamlit run storm. py, and then hit enter, and there we go. It looks like it may have worked on the first try, and here you can see the streamlit server is up and running. We have it on Local Host. It is still trying to spin up, so maybe it didn't work, but we'll see. Okay, so it does look like it worked. Let's start your first research, and then we'll we type dogs, and then we'll do research, and we should be able to see all the logs happening here. So it says brainstorming open it, start identifying different perspectives for researching this topic, and there we go. We can see the HTTP request to the OpenAI endpoint, so it looks like we didn't run into any

bugs this time. Maybe they updated the code since the last time I pulled, which was, I think, yesterday. But good for them. This should work for you. Here we go. It's doing all of the browsing. Look how many different pages it is browsing so fast. Okay, so the article finished. Let's see. Show final article. And here we go. All the information we could ever want. Run locally and it did all the research in real time. This is the most real-time information it could have possibly pulled. So we have taxonomy and evolution, physical characteristics, behavior, all very well organized, and references to each of the articles that it got the facts from. Over here, we can actually select the references and it'll link directly to it and give us a highlight of each of the articles. It is such a good researcher. Okay, but now what if you don't want to hit open A's API? And I guess technically you could probably set up your own web scraper too, although that's a pretty big thing to accomplish. So you can get all of this.

except for the web scraping running locally. If you want, and currently they support vlm with mistol, and they actually have an example of how to do it. But I haven't set it up myself, and I've seen a GitHub issue where they talk about getting o Lama working, but again I couldn't really get it working. I didn't try that hard. If you want to see me do a full local tutorial of this, just let me know in the comments. If you enjoyed this video, please consider giving a like and subscribe, and I'll see you in the next one.

STORM AI: Create Full Wikipedia Pages in Minutes With AI Agents (Tutorial)

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Matthew Berman

- The Stanford AI research project, Storm, is an AI researcher that can generate Wikipedia pages on various topics by referencing information from different websites.

- It can be run locally, is open source, and has a demo page available for testing, with a planned human-AI collaboration mode.

- The project, known as Storm, has a GitHub page with a large number of stars, and users can encounter some bugs when trying to run it locally, but these can be fixed.

Setting Up Storm Repository and Python Environment

The process involves cloning the Storm repository from the GitHub page and creating a Python 3.

After activating the Storm environment, the next steps include installing the requirements and setting up a secrets.

The process also involves opening the Storm folder in VS code and following the instructions to initialize the project.

To use the OpenAI API, you need to obtain an OpenAI API key from your OpenAI account's platform.

com search API also requires a key, which can be obtained from the u.

Both APIs can be integrated using command line or a simple UI for ease of use.

Setting Up Streamlit and Running the Demo

The process involves copying the file to the Streamlit folder and running the Streamlit server.

The demo includes researching a topic, with the ability to see all the logs and the final article in real time.

It also showcases the option of setting up a web scraper as an alternative to using OpenAI's API.

Difficulty with Setting Up Web Scraping Locally

Setting up web scraping locally can be challenging, especially with the VLM and Mistol support.

There are examples available, but some users have difficulty getting it to work, as discussed in GitHub issues.

Further support and tutorials may be needed for successfully implementing local web scraping.