Gato: 멀티모달 학습을 위한 멀티태스킹 모델

11억 개의 매개 변수가 있는 GPT와 유사한 모델인 Gato는 텍스트, 컴퓨터 비전, 아타리 게임 등 600개의 작업을 학습합니다.

### Gato: 텍스트, 컴퓨터 비전 및 제어를 위한 멀티태스킹 AI 모델
* 🤖   Gato는 약 **11억 개의 파라미터**를 가진 GPT와 유사한 모델로, 약 **600개의 다양한 작업**을 학습할 수 있으며 아타리 게임을 플레이하는 **평균적인 인간**보다 뛰어난 성능을 발휘합니다.
* 🔤   이 모델은 다양한 작업과 양식을 일련의 벡터로 변환하고 이를 사용하여 다음 토큰을 **예측**함으로써 다양한 양식을 **결합**하는 강력한 엔지니어링 아이디어를 보여줍니다.
* 🖼️   텍스트, 이미지, 제어 입력을 처리하여 공유 공간으로 변환하고 동작을 위한 토큰을 출력할 수 있으며, 비디오 게임과 로봇 공학을 통해 **멀티모달 기능과 구현**을 보여줄 수 있습니다.
* 🎮   Gato는 약 590개의 제어 데이터 세트에 대해 학습되었으며, 대부분의 작업에서 무작위 확률보다 높은 점수를 얻었고 일부 작업에서는 전문가보다 더 높은 점수를 얻어 **폭넓은 능력**을 보여주었습니다.
* 🔣   모델의 토큰화 및 임베딩 메커니즘은 텍스트, 이미지, 불연속형 값에 따라 다르므로 단일 모델에서 다양한 양식을 **효과적으로 결합**할 수 있습니다.
* 🌐   가토의 다음 단계는 **모델 확장**, **작업 간 전이 학습**, 특정 사용 사례에 대한 기능 활용 등이며, 이는 멀티태스킹 AI 모델 분야의 **흥미로운 발전**을 나타냅니다.

could you train one machine learning model to solve hundreds of tasks? Could it learn text tasks as well as computer vision, but also control tasks like play Atari or control robotics? The Gato paper from DeepMind shows that you can do this, and with a model that's maybe simpler and smaller than you might think. In this video, we'll talk about Gato, which is a GPT-like model, not so many parameters, just about a billion, 1.1 billion parameters, that was able to learn about 600 different tasks and play Atari better than the average human for a lot of games. [Music] Let's look at the paper and some of the interesting thoughts that it discusses. So, this is the paper, a generalist agent, Gato. We look at some of the figures, just to scan them first. So, a single model does a lot of different tasks. Here, it controls robots. Here, it takes an image and labels it, so image captioning. Here, it's having a dialogue with an agent or a human, like, you know, controlling playing Atari and doing a bunch of

things so one model not super massive. It does a lot of things. How it does this is explained in figure two and we'll get into this a little bit more. So all of these different tasks and modalities are translated into sequences of vectors, and then the model uses them to predict the next token. And this is really one of the major ideas here. So which is for me more of an engineering idea than let's say a theory idea, which is what is a good tokenization in let's say embedding scheme for not only text but also images, images plus text for playing Atari games, for doing things with robots. So this is one of the ideas that really gives the engineering tools to think about how to bring all of these modalities together. So the model is a single GPT model, and then a lot of it goes into the tokenization, how you tokenize the various modalities. So they go into how to tokenize text, images. We'll get into that in a little bit. One thing that is interesting here is the many modalities.

that the model is able to deal with. I've been thinking about, you know, how can we, what is a good visual for us to capture to discuss, uh, modalities that models accept either as input or as output. I have a kind of visual here. Let me guide you through it so we can also see how this works or in relation to other previous models. So on the left, you have the inputs and then on the right, you have the output. With a GPT-like model, the input is text and the output is text, so it generates text and that's what comes out on the other side. If you're familiar with BERT, BERT really maybe generates text embeddings or token embeddings that can be turned into a text embedding. And then downstream from that, these can be used to train classifiers or do semantic search or do other things. Something like computer vision models encode representation for images, which can be thought of as embeddings. GANs, not really sure what goes into them. There's random noise that goes in but

they output images. We know that at least for sure if you've been following the incredible things happening within the generations between Dale Imogen party from Google stable diffusion is the most recent one coming out this month. These are models that take text and can take images as well, and they output an image on the other side. Something like clip, which is maybe foundational to these image generation models or the ideas that packages are foundational. It can take a piece of text and generate a text embedding, or they can take an image and generate an image embedding. The interesting thing here is that it posts them or like sends them or translates them into a shared space where text describing a dog will be close to images about dogs, and sort of having that space sort of translated between text and images is very important for those Technologies now. Something like gato does a little bit more of this, so we can take text, it can take images, it can take control input so this is Atari, but also it's robotic sensors and robotic controls on the other side. It outputs

text is not an image generation model, so it does not do that yet. But it outputs tokens, which can be actions on Atari, or actions to let's say move a robot somehow. And the key thing that we discussed here is that it just translates all of this, translates all of this into this sequence-to-sequence modality or of data. So everything on the tech side is a token, everything on images can be a token, but also Atari playing and robotic sensors are also translated into sequences. And the model, you know, predicts the next token in this way. And so that's one of the main ideas that I think are great takeaways from this moment in multimodal research. So not all of the ideas that we will be discussing here are, you know, invented from scratch by Gato, but it's a useful point to have as a synthesis of everything else that is happening. In sort of a culmination of all of the work that a lot of different teams are doing.

bring the modalities together. We can look at, so we looked at a couple of the figures. So this is the idea of this, is the, you know, figure three talks about how the inputs and outputs are handled or given to the model. So you can give the model, optionally, a prompt to sort of set it in the right mode for what modality you're thinking about or you're processing right now. And then the sequence, some of it is observations. So these are like the image on the screen. So these go into the model. And then the model predicts what is a good action token. So what is a good action on the controller in response to this. And sort of this back and forth is how the model translates something like action or like controlling an Atari game, which is really a form of embodiment. If you look back to a previous video on experience grounds language, that's really one of the main ideas that makes gato very interesting for me, which is it brings together not only multimodality but also a form of embodiment through video games but also through Robotics.

and it brings forth the mechanisms for you know how to do that tokenization. These are the data sets, so about eight data sets that are, let's say, vision and language, and then about 590 different control data sets, so video games. And we'll, we'll talk a little bit about how the video game data is translated into sequences. And I think let's go over the figures a little bit. So this is a little bit about the robotics task and how it's structured. So there are three shapes, and the model controls a robotic arm trying to stack one over the other. I think, so stack the red on Blue, ignoring green. And there are different shapes for the different ones. And you know, if you learn one and two shape sets, one and two, will the model be able to do three? Well, let's transfer without training it specifically on three. And that's one of the main ideas here about how much transfers between tasks, and they talk about it a little bit in the paper. This is a pretty dense.

figure took me a while to really understand, but it's a way to evaluate the maybe 590 different tasks and to understand that I have. Let's say this sort of breakdown here. So let's ignore all of the figure. Let's look at it. Just point zero here. So at point zero, we have about 570. That's just me reading sort of how high that is. So that is the number of tasks where Gato scored better than random chance. So this is the one model trained on all of these tasks. It scores in the vast majority of them. It does better than random chance. Now the point that they mentioned about this figure is this number 450, which is to say that there are 450 tasks where Gato scored more than 50 percent of the expert score. So how this is evaluated is that you have these 592 tasks. Each task, there's an expert. So either in a reinforcement learning model or a human that is really good at that task. And then this is a measure of how close or

far that that is this is a visual of me sort of trying to put this visually so you have you know 592 tasks. You have an expert for each one. The expert scores let's say 100 and like gato this one model learns all of them and scores let's say different tasks but then gato also so just to add one thing to this figure so there are cases where it does better than than the expert. So 100 is you know it's not let's say solving the actual task. 100 could be you know getting 5 000 points in Pac-Man or something and that's what we see here. So there are eyeballing it about 100 tasks where gato is able to do more than 100. So this is just my read of this figure and sort of trying to understand it another thing is about how the these training data was sourced. So for these control tasks the experts play the games. We have let's say recordings of them episodes of The Tokens of the model

interacting or the expert interacting with that so during the training phase gato is not plugged into an Atari environment in playing Pacman gather is actually trained on pre-recorded sequences of the expert but then let later it's evaluated by being plugged into the environment and doing the actions it was trained on and this this sort of makes sense as an approach one of the key ideas for me to also wrap my head around that I think is very useful is the tokenization schemes and the tokenization and like embedding mechanism used in in gato so text is embedded in tokens just like GPT just like these other models you have words you break them down into or is there sub word tokens and then you generate tokens that are then mapped into sort of words images are done in a little bit of a different way so you have an image it splits it into patches and then passes every patch through a resnet so you have let's say multiple vectors and then you add a positional embedding and then that

let's say that set of vectors is their representation of an image. So, one image is not one vector. One image is like, I don't know, 16 vectors or is it 16 by 16? So, it just breaks it down into, I think, four by four. We'll look at the paper for more, sort of, uh, but it's like, let's say, multiple, um, multiple vectors, 16 by 16 patches. And so, that takes care of these two modalities, so text and images. But before moving on to the other modalities, this opens up the door, uh, for us to think about how to use both images and text together. So, image captioning, for example, is this case where you have an image and some text. And so, you can have a sequence of these vectors from the representing the image and then the vectors representing the sequence and representing the text. And then that sequence combines these two modalities. The third one is discrete values. This is the one they use for training Atari, let's say, controller inputs for capturing Atari controller inputs there.

aren't very many details. I haven't worked with let's say these environments personally myself, but I think let's say one way I understand it is kind of like this. So they assigned about a thousand tokens to these actions to me. One way of doing that would be the state of the controller, so where the joystick is pointing, and maybe the state of the button. This could also be different vectors. I think these are implementation details, but let's not get too bogged down here. But then every action can be turned into a token, and the model learns the best embeddings from the training set. This is an example of, you know, doing the Hadoken, I guess. So step one, press nothing, then down, then down and right, then right and button. You'll look up into the file for all of this, or let's say the index of the embeddings. You get the token IDs, and then you get the tokens. This is almost exactly what we do with text and token embeddings. And then another idea is that playing video games is not just a sequence of controls.

need an image also so you need this observation of the image or the state of the game of the screen. So if you have three steps of playing Pac-Man, there's an image and an action with each one. You kind of do it like this where you have these 16 by 16, let's say, vectors representing the image. And then after them, you have the action of the button here. So that's time step one. Then time step two, you have the observation or the image vectors in yellow. And then the action in green. And so that's how you pack the various observations and actions. And that's how the model is trained. This is a look at how Gato brings together all of these modalities, multimodalities, but also embodiment and language modeling. It's a very interesting approach about thinking about all these modalities as they converge into a single model. Now, next steps are definitely. So we know scale is one of them. So this is a small model that was able to do all of these things.

what if you expand that? What if you give it 100 billion parameters to learn from? What if you throw massive amounts of data? What is it going to be able to do and what kinds of that's a positive transfer over earning this one task improves this, this other task? That's a clear sort of next, uh, step and next, yeah, step in the process of inquiry into the value of these models. I'm really interested in seeing, you know, for specific use cases, how to get modality. So, I really want, let's say, audio and the text models that are traded on both, so that they can maybe comment or describe music. So, there are, yeah, many use cases that you can think of but also between just images and text, uh, there's, there's so much that can be done that this opens the, the door to of thinking of them as, uh, yeah, we can have a sequence of three different images with text between them and then we can very easily also not very, not very easily but we can comprehend the ideas of image generation.

and plugging that into sort of the fold. So it's an exciting field. I definitely think there will be a lot more in these areas coming up in the next few months. So stay tuned. I hope you've enjoyed this.

A Generalist Agent (Gato) - DeepMind's single model learns 600 tasks

Jay Alammar

가토: 600개 이상의 능력을 갖춘 다재다능한 GPT형 모델

Gato는 텍스트, 컴퓨터 비전, 아타리 게임, 로봇 제어 등 600가지가 넘는 다양한 작업을 수행할 수 있는 11억 개의 매개변수가 있는 GPT와 유사한 모델입니다.

이 모델은 다양한 작업과 양식을 일련의 벡터로 변환하고 이를 사용하여 다음 토큰을 예측함으로써 이를 달성하며, 다양한 양식을 위한 토큰화의 세심한 엔지니어링이 필요합니다.

Gato의 다재다능함은 여러 양식을 입력과 출력으로 처리할 수 있는 능력에 있으며, 다양한 작업을 위한 강력하고 유연한 모델입니다.

가토의 멀티모달 연구가 미친 영향

가토의 최신 모델인 안정적 확산은 텍스트와 이미지를 처리하여 공유 공간에서 해당 출력을 생성함으로써 두 양식 사이의 간극을 메울 수 있습니다.

이미지 생성 모델과 달리 가토의 모델은 입력을 시퀀스 간 양식으로 변환하여 아타리 및 로봇 제어에서 동작에 사용할 수 있는 토큰을 생성하는 데 중점을 둡니다.

이 연구는 텍스트, 이미지, 아타리 게임플레이, 로봇 센서 등 다양한 양식을 결합하여 궁극적으로 비디오 게임과 로봇 공학을 통한 구현 형태를 통합하는 데 있어 중요한 진전을 이루었습니다.

시각, 언어 및 로봇 공학 분야의 학습 전달

이 연구는 시각, 언어, 로봇 공학 등 다양한 과제 간의 학습 전이에 대해 논의합니다.

590개의 제어 과제에서 단일 모델이 어떻게 570개의 과제에서 무작위보다 더 높은 점수를 얻는지 살펴봅니다.

이 연구는 전문가 점수와 비교하여 모델의 성능을 평가하며, 약 100개의 과제에서 모델이 전문가를 능가하는 성능을 보였습니다.

Gato의 교육 접근 방식과 토큰화

Gato는 미리 녹음된 전문가 시퀀스에 대해 학습한 다음 아타리 환경에서 평가됩니다.

Gato의 토큰화는 텍스트를 토큰으로 분해하고 이미지를 패치로 분할한 후 리셋을 통과시켜 텍스트와 이미지를 함께 사용할 수 있도록 합니다.

Gato는 또한 Atari 컨트롤러 입력을 훈련하기 위해 불연속 값을 사용합니다.

토큰을 사용하여 게임 및 언어 모델링에서 행동과 관찰을 표현하는 것은 멀티모달 학습의 핵심 요소입니다.

모델은 텍스트 및 토큰 임베딩에 사용되는 프로세스와 유사하게 훈련을 통해 각 동작에 가장 적합한 임베딩을 학습합니다.

이러한 맥락에서 멀티모달 학습은 이미지 관찰을 특정 행동과 결합하여 학습을 위한 시간 단계로 묶는 것을 포함합니다.

Gato의 접근 방식은 다양한 양식과 구현을 단일 모델에 통합하며, 확장성은 다음 단계의 중요한 요소입니다.

언어 모델 확장이 작업 성능에 미치는 영향

파라미터를 최대 1,000억 개까지 확장하고 데이터 입력을 늘리면 언어 모델이 여러 작업에서 지식을 학습하고 전달하는 능력이 크게 향상될 수 있습니다.

이러한 확장은 오디오 및 텍스트와 같은 양식을 이해하고 생성하는 모델의 능력을 향상시켜 음악 설명, 텍스트에서 이미지 생성 등 다양한 잠재적 사용 사례를 열어줄 수 있습니다.

앞으로 이러한 영역에서 흥미로운 발전이 이루어질 예정이니 더 많은 발전을 기대해 주세요.