Gemini 2.0 Flash를 통한 무료 오픈 소스 보안 컴퓨터 사용: AI 에이전트가 Linux 데스크톱을 제어합니다.

### Gemini 2.0 기반 무료 오픈 소스 컴퓨터 제어 에이전트
* 💻 이 동영상에서는 무료 오픈 소스 에이전트인 '보안 컴퓨터 사용'을 소개합니다. 이 에이전트는 e2b 클라우드 VM 내에서 컴퓨터를 제어하는 것으로, Anthropic의 컴퓨터 사용과 유사하지만 훨씬 저렴합니다.
* 💡 Anthropic의 모델을 사용하는 원본과 달리 이 버전은 OS Atlas, Llama 2 3B와 같은 오픈 소스 모델을 활용하며 비전 및 LLM 작업을 위해 두 개의 공급자(OpenAI 및 Firework)가 필요합니다.
* ⚙ 이 튜토리얼에서는 무료 Gemini 2.0 Flash를 사용하도록 에이전트를 구성하는 방법을 보여드리며, FFmpeg 및 Poetry 설치, 리포지토리 복제, `os_computer_use` 브랜치 확인, API 키로 LLM Python 파일 및 `.env` 파일 수정이 필요합니다.
* 🏃 ↪f_200D↩♂️ 설정은 'poetry install'을 통해 패키지를 설치하고 'poetry start'로 에이전트를 실행하여 데스크톱 환경의 라이브 스트림을 표시하는 것입니다.
* 🔍 이 에이전트는 Google에서 '고양이'를 검색하고 계산기를 사용하는 등의 작업을 성공적으로 수행하여 브라우저 전용 대안과 달리 로컬 애플리케이션과 상호 작용할 수 있는 능력을 보여주었습니다.
* ⚠ 에이전트와 로컬 앱의 상호 작용은 기능적이지만 Anthropic에 비해 덜 세련되고 때때로 종료되지만 여전히 유용한 무료 대안입니다.
* ✨ 이 에이전트는 100만 개의 컨텍스트 창을 지원하므로 여러 단계와 심층적인 이해가 필요한 복잡한 작업도 처리할 수 있습니다.
* 👍 이 영상은 에이전트의 자유로운 특성, 강력한 비전 기능, 향후 발전 가능성을 강조하며 클로즈드 소스 옵션에 대한 매력적인 오픈 소스 대안으로서의 역할을 강조하며 마무리됩니다.

보안 컴퓨터 사용은 다양한 작업을 위해 Linux 데스크톱을 제어하는 무료 오픈 소스 AI 에이전트를 제공합니다.

[Music] [Applause] Hi, welcome to another video. So, Computer Use came out some time ago and worked pretty well, although it has been a lot more expensive to use. So, as with anything, multiple open-source projects came out that try to do the same and do it much better than Claude and make it quite cool to use.

But there's another agent that I saw, and it's this Secure Computer Use one. This is by e2b, and what it basically does is that it allows you to do what Anthropic's Computer Use does, but in a cloud VM that is given by e2b.

If you know e2b, then you'll know that they are widely used for their code interpreter sandbox thing, but they also recently launched their whole virtual sandbox thing for agents that want to control a computer.

So it basically gives you AI agents with Linux desktops that they can control and use to do anything. Now, this is an implementation of that by them, and if you'll see this, then you can see that this is based on the main Anthropic Computer Use thing, which means that it will be very token-heavy.

But actually, there's another version of this implementation which you can view by changing the Git branch here to OS Computer Use. Here you'll see that the implementation in this branch is different, and this is based on open-source models instead of Anthropic's model. It uses the same sandbox, but it also uses OpenAI's and Firework's as well.

Now you may ask why it needs two providers. Well, that's a good question, and I don't know why either. From what I was able to understand from the code, it uses three models: the first one is the OS Atlas model.

If you don't know about OS Atlas, then it's an open-source vision model that basically is trained to give...

It outputs pixel addresses and controls.  Its output is like a computer using pixel coordinates

to click, type, or do anything like that. But to control this, you need a proper LLM that can be asked to give the coordinates for

the tasks. So for that, it uses Llama 2 3B via Firework, as it's a great open-source model. But as you know, it

cannot do vision things. So for vision help, it uses Llama 2 to analyze images as well, via OpenRouter. So that's why it

needs two providers. But I'll tell you how you can get better results than this by changing the models to Gemini 2.0

Flash and using it technically for free. So let's get right into it and let me show you how you can make it work.

First of all, you'll need to install the stuff that it needs, which is FFmpeg and Poetry. So just get it installed by

running this command. Once done, clone the repo onto your computer. Now change the directory to 'secure_computer_use'. Once done, we'll need

to change the branch to the 'os_computer_use' branch by running the `git checkout` command. This will get the branch

changed to it. Now, first let me tell you how you can configure it to use Gemini 2.0 Flash,

because I like using it with that since it's free. So to do that, just go to this LLM

Python file. Here, first change the base URL in this to the Gemini OpenAI compatible endpoint, and in this also

change it to OpenAI compatible like this. Once that's done, also update this line to correctly work with the

OpenAI SDK. Once that's done, just go back and create an .env file. Now here, enter the

Ollama API key that you can get for free from their site. And there's no apparent limit as well, so just get it. Once done,

Just enter the Gemini API key in both variables. Once done, we can start using it. So, first of all, to use it the

first time, you'll need to make sure that you install every package by running the `poetry install` command. Then, once that's done, just run

it with `poetry start`, and it will get started. Now, this will start it, and you'll see the e2e sandbox open up here.

This is a live stream of the desktop, so you cannot click and do stuff like that. Anyway, now here you can just enter what

you want it to do. Let me show you something simple, which is also shown in their demo, and that's to search for cats

on Google. So let's ask it here. Once we do that, you'll see that it starts doing it. The live stream here is a bit delayed

than the prints on the terminal that you see, so there's that. But it's in a proper, safe, sandboxed environment, which is a

great thing. So you can see that it's doing it here. Let's wait a bit. [Music]

And it's now done. So it did this quite well, which is pretty great. Another thing that I want to try

is that I wanted to use the calculator and make a simple calculation in it. Because the best part about it is that

it can use local apps, unlike Browser Use, which was another open-source project that was really very great but could

only use browsers. Anyway, it's now doing that. Let's wait a bit. [Music] And it's now done. So it can use the

local apps quite well, so that's great. I have tried it on some other stuff as well, and it works

well. Although, because it uses local apps, it's not as polished as its Anthropic counterpart because it terminates

sometimes and stuff like that, which is fine considering that we are using it all for free. This is a great

implementation, and you can surely extend it even more if you want, as the code is also very simple to understand and

extend. Navigation is a challenging thing using it with Gemini 2.  It's also great because it's free and has really

great vision capabilities, and it also has a quite big, one-million-context window, which means that it can also do some big tasks that

may require more clicks or a deeper understanding of something. So it works pretty well and is super cool to use, and it's a really

great open-source version of the Anthropic Claude thing. It's great to see some new alternatives to closed

source things popping up that people can utilize in different ways. Overall, it's pretty cool. Anyway, let me know your thoughts in

the comments. If you liked this video, consider donating to my channel through the Super Thanks option below, or you can

also consider becoming a member by clicking the Join button. Also, give this video a thumbs up and subscribe to my channel.

I'll see you in the next video. Till then, bye. [Music]

E2B Gemini 2.0 Computer Use: This FULLY FREE Agent can Control Whole Computer & DO ANYTHING!

Facing the future with AI! 

Here you'll find content about multiple AI Tools that are actually useful and sometimes free.

AICodeKing

안전한 컴퓨터 사용 및 기능 소개

Anthropic의 컴퓨터 사용에 대한 오픈소스 대안인 안전한 컴퓨터 사용에 대해 소개합니다.

보안 컴퓨터 사용은 e2b 클라우드 가상 머신 내에서 작동하여 AI 에이전트에게 제어용 Linux 데스크톱을 제공합니다.

E2b의 가상 샌드박스를 통해 AI 에이전트는 컴퓨터를 제어하고 사용하여 다양한 작업을 수행할 수 있습니다.

초기 구현은 앤트로픽의 컴퓨터 사용을 기반으로 하므로 토큰이 많이 사용됩니다.

다른 구현 방식은 OS Atlas, Llama 2, OpenRouter와 같은 오픈 소스 모델을 사용합니다.

안전한 컴퓨터 사용에 사용되는 모델에 대한 설명

오픈 소스 비전 모델인 OS Atlas는 컴퓨터 상호작용을 위한 픽셀 주소와 컨트롤을 출력합니다.

Firework를 통한 Llama 2 3B는 작업 조정 및 좌표 생성을 위한 LLM을 제공합니다.

Llama 2는 OpenRouter를 통한 이미지 분석에도 사용됩니다.

비전과 LLM 기능의 조합으로 인해 두 개의 제공업체가 필요합니다.

Gemini 2.0 플래시를 사용하면 향상된 결과를 얻을 수 있습니다.

안전한 컴퓨터 사용을 위한 설정 및 구성 지침

리포지토리를 복제하고 'os_computer_use' 브랜치로 변경합니다.

Gemini 2.0 플래시를 사용하도록 LLM Python 파일 구성하기.

Ollama 및 Gemini API 키로 .env 파일 만들기.

시를 사용하여 패키지를 설치하고 애플리케이션을 실행합니다.

데스크톱 환경의 라이브 스트리밍 데모.

예제 작업: Google에서 '고양이' 검색하기.

계산을 위해 로컬 계산기 애플리케이션을 사용합니다.

애플리케이션의 성능과 한계를 평가합니다.

Gemini 2.0과 대형 컨텍스트 창 사용의 장점을 강조합니다.

훌륭한 오픈소스 대안으로 애플리케이션을 전반적으로 평가합니다.

시청자가 자신의 생각을 공유하고 채널을 응원하도록 장려합니다.