[Music] [Applause] Hi, welcome to another video. So, Computer Use came out some time ago and worked pretty well, although it has been a lot more expensive to use. So, as with anything, multiple open-source projects came out that try to do the same and do it much better than Claude and make it quite cool to use.

But there's another agent that I saw, and it's this Secure Computer Use one. This is by e2b, and what it basically does is that it allows you to do what Anthropic's Computer Use does, but in a cloud VM that is given by e2b.

If you know e2b, then you'll know that they are widely used for their code interpreter sandbox thing, but they also recently launched their whole virtual sandbox thing for agents that want to control a computer.

So it basically gives you AI agents with Linux desktops that they can control and use to do anything. Now, this is an implementation of that by them, and if you'll see this, then you can see that this is based on the main Anthropic Computer Use thing, which means that it will be very token-heavy.

But actually, there's another version of this implementation which you can view by changing the Git branch here to OS Computer Use. Here you'll see that the implementation in this branch is different, and this is based on open-source models instead of Anthropic's model. It uses the same sandbox, but it also uses OpenAI's and Firework's as well.

Now you may ask why it needs two providers. Well, that's a good question, and I don't know why either. From what I was able to understand from the code, it uses three models: the first one is the OS Atlas model.

If you don't know about OS Atlas, then it's an open-source vision model that basically is trained to give...

It outputs pixel addresses and controls.  Its output is like a computer using pixel coordinates

to click, type, or do anything like that. But to control this, you need a proper LLM that can be asked to give the coordinates for

the tasks. So for that, it uses Llama 2 3B via Firework, as it's a great open-source model. But as you know, it

cannot do vision things. So for vision help, it uses Llama 2 to analyze images as well, via OpenRouter. So that's why it

needs two providers. But I'll tell you how you can get better results than this by changing the models to Gemini 2.0

Flash and using it technically for free. So let's get right into it and let me show you how you can make it work.

First of all, you'll need to install the stuff that it needs, which is FFmpeg and Poetry. So just get it installed by

running this command. Once done, clone the repo onto your computer. Now change the directory to 'secure_computer_use'. Once done, we'll need

to change the branch to the 'os_computer_use' branch by running the `git checkout` command. This will get the branch

changed to it. Now, first let me tell you how you can configure it to use Gemini 2.0 Flash,

because I like using it with that since it's free. So to do that, just go to this LLM

Python file. Here, first change the base URL in this to the Gemini OpenAI compatible endpoint, and in this also

change it to OpenAI compatible like this. Once that's done, also update this line to correctly work with the

OpenAI SDK. Once that's done, just go back and create an .env file. Now here, enter the

Ollama API key that you can get for free from their site. And there's no apparent limit as well, so just get it. Once done,

Just enter the Gemini API key in both variables. Once done, we can start using it. So, first of all, to use it the

first time, you'll need to make sure that you install every package by running the `poetry install` command. Then, once that's done, just run

it with `poetry start`, and it will get started. Now, this will start it, and you'll see the e2e sandbox open up here.

This is a live stream of the desktop, so you cannot click and do stuff like that. Anyway, now here you can just enter what

you want it to do. Let me show you something simple, which is also shown in their demo, and that's to search for cats

on Google. So let's ask it here. Once we do that, you'll see that it starts doing it. The live stream here is a bit delayed

than the prints on the terminal that you see, so there's that. But it's in a proper, safe, sandboxed environment, which is a

great thing. So you can see that it's doing it here. Let's wait a bit. [Music]

And it's now done. So it did this quite well, which is pretty great. Another thing that I want to try

is that I wanted to use the calculator and make a simple calculation in it. Because the best part about it is that

it can use local apps, unlike Browser Use, which was another open-source project that was really very great but could

only use browsers. Anyway, it's now doing that. Let's wait a bit. [Music] And it's now done. So it can use the

local apps quite well, so that's great. I have tried it on some other stuff as well, and it works

well. Although, because it uses local apps, it's not as polished as its Anthropic counterpart because it terminates

sometimes and stuff like that, which is fine considering that we are using it all for free. This is a great

implementation, and you can surely extend it even more if you want, as the code is also very simple to understand and

extend. Navigation is a challenging thing using it with Gemini 2.  It's also great because it's free and has really

great vision capabilities, and it also has a quite big, one-million-context window, which means that it can also do some big tasks that

may require more clicks or a deeper understanding of something. So it works pretty well and is super cool to use, and it's a really

great open-source version of the Anthropic Claude thing. It's great to see some new alternatives to closed

source things popping up that people can utilize in different ways. Overall, it's pretty cool. Anyway, let me know your thoughts in

the comments. If you liked this video, consider donating to my channel through the Super Thanks option below, or you can

also consider becoming a member by clicking the Join button. Also, give this video a thumbs up and subscribe to my channel.

I'll see you in the next video. Till then, bye. [Music]

E2B Gemini 2.0 Computer Use: This FULLY FREE Agent can Control Whole Computer & DO ANYTHING!

Facing the future with AI! 

Here you'll find content about multiple AI Tools that are actually useful and sometimes free.

E2B Gemini 2.0 Computer Use: This FULLY FREE Agent can Control Whole Computer & DO ANYTHING!

Scripts