[Music] [Applause] Hi, welcome to another video. So, Computer Use came out some time ago and worked pretty well, although it has been a lot more expensive to use. So, as with anything, multiple open-source projects came out that try to do the same and do it much better than Claude and make it quite cool to use.

But there's another agent that I saw, and it's this Secure Computer Use one. This is by e2b, and what it basically does is that it allows you to do what Anthropic's Computer Use does, but in a cloud VM that is given by e2b.

If you know e2b, then you'll know that they are widely used for their code interpreter sandbox thing, but they also recently launched their whole virtual sandbox thing for agents that want to control a computer.

So it basically gives you AI agents with Linux desktops that they can control and use to do anything. Now, this is an implementation of that by them, and if you'll see this, then you can see that this is based on the main Anthropic Computer Use thing, which means that it will be very token-heavy.

But actually, there's another version of this implementation which you can view by changing the Git branch here to OS Computer Use. Here you'll see that the implementation in this branch is different, and this is based on open-source models instead of Anthropic's model. It uses the same sandbox, but it also uses OpenAI's and Firework's as well.

Now you may ask why it needs two providers. Well, that's a good question, and I don't know why either. From what I was able to understand from the code, it uses three models: the first one is the OS Atlas model.

If you don't know about OS Atlas, then it's an open-source vision model that basically is trained to give...

It outputs pixel addresses and controls.  Its output is like a computer using pixel coordinates

to click, type, or do anything like that. But to control this, you need a proper LLM that can be asked to give the coordinates for

the tasks. So for that, it uses Llama 2 3B via Firework, as it's a great open-source model. But as you know, it

cannot do vision things. So for vision help, it uses Llama 2 to analyze images as well, via OpenRouter. So that's why it

needs two providers. But I'll tell you how you can get better results than this by changing the models to Gemini 2.0

Flash and using it technically for free. So let's get right into it and let me show you how you can make it work.

First of all, you'll need to install the stuff that it needs, which is FFmpeg and Poetry. So just get it installed by

running this command. Once done, clone the repo onto your computer. Now change the directory to 'secure_computer_use'. Once done, we'll need

to change the branch to the 'os_computer_use' branch by running the `git checkout` command. This will get the branch

changed to it. Now, first let me tell you how you can configure it to use Gemini 2.0 Flash,

because I like using it with that since it's free. So to do that, just go to this LLM

Python file. Here, first change the base URL in this to the Gemini OpenAI compatible endpoint, and in this also

change it to OpenAI compatible like this. Once that's done, also update this line to correctly work with the

OpenAI SDK. Once that's done, just go back and create an .env file. Now here, enter the

Ollama API key that you can get for free from their site. And there's no apparent limit as well, so just get it. Once done,

Just enter the Gemini API key in both variables. Once done, we can start using it. So, first of all, to use it the

first time, you'll need to make sure that you install every package by running the `poetry install` command. Then, once that's done, just run

it with `poetry start`, and it will get started. Now, this will start it, and you'll see the e2e sandbox open up here.

This is a live stream of the desktop, so you cannot click and do stuff like that. Anyway, now here you can just enter what

you want it to do. Let me show you something simple, which is also shown in their demo, and that's to search for cats

on Google. So let's ask it here. Once we do that, you'll see that it starts doing it. The live stream here is a bit delayed

than the prints on the terminal that you see, so there's that. But it's in a proper, safe, sandboxed environment, which is a

great thing. So you can see that it's doing it here. Let's wait a bit. [Music]

And it's now done. So it did this quite well, which is pretty great. Another thing that I want to try

is that I wanted to use the calculator and make a simple calculation in it. Because the best part about it is that

it can use local apps, unlike Browser Use, which was another open-source project that was really very great but could

only use browsers. Anyway, it's now doing that. Let's wait a bit. [Music] And it's now done. So it can use the

local apps quite well, so that's great. I have tried it on some other stuff as well, and it works

well. Although, because it uses local apps, it's not as polished as its Anthropic counterpart because it terminates

sometimes and stuff like that, which is fine considering that we are using it all for free. This is a great

implementation, and you can surely extend it even more if you want, as the code is also very simple to understand and

extend. Navigation is a challenging thing using it with Gemini 2.  It's also great because it's free and has really

great vision capabilities, and it also has a quite big, one-million-context window, which means that it can also do some big tasks that

may require more clicks or a deeper understanding of something. So it works pretty well and is super cool to use, and it's a really

great open-source version of the Anthropic Claude thing. It's great to see some new alternatives to closed

source things popping up that people can utilize in different ways. Overall, it's pretty cool. Anyway, let me know your thoughts in

the comments. If you liked this video, consider donating to my channel through the Super Thanks option below, or you can

also consider becoming a member by clicking the Join button. Also, give this video a thumbs up and subscribe to my channel.

I'll see you in the next video. Till then, bye. [Music]

E2B Gemini 2.0 Computer Use: This FULLY FREE Agent can Control Whole Computer & DO ANYTHING!

Facing the future with AI! 

Here you'll find content about multiple AI Tools that are actually useful and sometimes free.

AICodeKing

안전한 컴퓨터 사용 및 기능 소개

Anthropic의 컴퓨터 사용에 대한 오픈소스 대안인 안전한 컴퓨터 사용에 대해 소개합니다.

보안 컴퓨터 사용은 e2b 클라우드 가상 머신 내에서 작동하여 AI 에이전트에게 제어용 Linux 데스크톱을 제공합니다.

E2b의 가상 샌드박스를 통해 AI 에이전트는 컴퓨터를 제어하고 사용하여 다양한 작업을 수행할 수 있습니다.

초기 구현은 앤트로픽의 컴퓨터 사용을 기반으로 하므로 토큰이 많이 사용됩니다.

다른 구현 방식은 OS Atlas, Llama 2, OpenRouter와 같은 오픈 소스 모델을 사용합니다.

안전한 컴퓨터 사용에 사용되는 모델에 대한 설명

오픈 소스 비전 모델인 OS Atlas는 컴퓨터 상호작용을 위한 픽셀 주소와 컨트롤을 출력합니다.

Firework를 통한 Llama 2 3B는 작업 조정 및 좌표 생성을 위한 LLM을 제공합니다.

Llama 2는 OpenRouter를 통한 이미지 분석에도 사용됩니다.

비전과 LLM 기능의 조합으로 인해 두 개의 제공업체가 필요합니다.

Gemini 2.0 플래시를 사용하면 향상된 결과를 얻을 수 있습니다.

안전한 컴퓨터 사용을 위한 설정 및 구성 지침

리포지토리를 복제하고 'os_computer_use' 브랜치로 변경합니다.

Gemini 2.0 플래시를 사용하도록 LLM Python 파일 구성하기.

Ollama 및 Gemini API 키로 .env 파일 만들기.

시를 사용하여 패키지를 설치하고 애플리케이션을 실행합니다.

데스크톱 환경의 라이브 스트리밍 데모.

예제 작업: Google에서 '고양이' 검색하기.

계산을 위해 로컬 계산기 애플리케이션을 사용합니다.

애플리케이션의 성능과 한계를 평가합니다.

Gemini 2.0과 대형 컨텍스트 창 사용의 장점을 강조합니다.

훌륭한 오픈소스 대안으로 애플리케이션을 전반적으로 평가합니다.

시청자가 자신의 생각을 공유하고 채널을 응원하도록 장려합니다.

E2B Gemini 2.0 Computer Use: This FULLY FREE Agent can Control Whole Computer & DO ANYTHING!

E2B Gemini 2.0 Computer Use: This FULLY FREE Agent can Control Whole Computer & DO ANYTHING!

스크립트