GPT-4 미니: 비용 효율적이고 빠르며 향상된 언어 모델

비용 효율적인 새로운 모델인 GPT-4 Mini는 다른 GPT 모델에 비해 향상된 성능과 빠른 처리 속도를 제공합니다.

### GPT-4 Mini 소개: 비용 효율적인 고성능 언어 모델 소개
* 🆕 OpenAI는 대형 모델에 대한 비용 효율적인 대안으로 성능과 경제성 사이의 균형을 제공하는 **GPT-4 미니 모델**을 출시했습니다.
* 💻 GPT-4.5 모델은 최상위 유료 모델로 자리 잡았으며, **GPT-4.5의 사용량 제한을 초과한 사용자는 이제 GPT-4 미니를 무료**로 사용할 수 있습니다.
* 📊 GPT-4 미니는 이전 모델에 비해 **향상된 한국어 기능, 빠른 성능, 비용 효율성**으로 주목받고 있습니다.
* 💰 GPT-4 미니의 가격 구조는 최상위 모델보다 훨씬 낮으며, **100만 토큰의 가격은 0.3달러**로 **GPT-4.5보다 25배 저렴**합니다.
* 🔢 GPT-4 Mini는 더 작은 모델임에도 불구하고 **수학, 코딩, 다중 모델 추론 작업 등 다양한 벤치마크에서 강력한 성능**을 발휘하는 것으로 유명합니다.
* 🧠 AI 언어 모델의 진화 추세는 암기에 대한 의존도를 줄이면서 **추론 능력을 향상**시키는 데 초점을 맞춘 작지만 효율적인 모델로 변화하고 있습니다.
* 🛠 AI 연구자들은 **모델 크기, 학습 토큰, 데이터 품질 간의 균형**을 유지함으로써 언어 모델 개발의 성능과 효율성을 최적화하고 있습니다.
* 🔮 향후 AI 언어 모델의 방향은 **이미지, 사운드, 동영상과 같은 다른 양식을 통합**하여 텍스트 처리 이상으로 모델의 기능을 확장하는 것일 수 있습니다.

Hello, we have introduced a new model called GPT 4 Mini through the overnight open AI ES GPT 4 event. It is now available for use. Our usual model is GPT 4.5. For free users, after using GPT 4.5 a certain number of times, they will switch to a lower model, which is GPT 3.5, with lower performance. The previous GPT 4 model used to be a paid model, but now even the paid model is based on GPT 4.5. It is much faster than GPT 4 and especially has significantly improved Korean language capabilities. The performance of the Korean tokenizer has also improved, resulting in much faster performance. Today, the newly added model is the 4 Mini version, which is positioned as a cost-effective model. Keywords and models with enhanced performance are being introduced. If you look at the performance, the purple is GT 4.

Currently, the best one is the red one on the left, which is the 45 Mini, a newly added model this time. The rest are now small models: the fun Flash Cloud Haiku, and the existing cost-effective model, GPT 3.5 Turbo. When you look at the benchmark scores, of course, now there's the GPD 45 in purple. It excels in various areas and benchmarks. Generally, the old model, the 3.5 model, had good scores. Also, Google's fun Flash, the entropy's Cloud Haiku, although a smaller model towards the GPT 3.5 side, is cheap, fast, and better than the 35. That's how these models were positioned. However, now a new model has come out that surpasses that. So, if you look at it, although GPT 4 is naturally not as good, it is a much smaller model.

However, in terms of performance, it is the best among all models. Roughly speaking, if you look at it this way, it makes sense. Judging by what they are emphasizing, the ability to listen, think step by step, make inferences, and think logically is very good. Especially in math and coding, in fact, large models do well, while small models cannot do anything. In a way, this is the prevailing theory. However, they have significantly improved the performance in that area, and multi-model reasoning is also possible. GPT-4 is not just a model that handles text, it stands for Generative Pre-trained Transformer, so it can also receive image inputs. Moreover, in the future, it is said that it will be able to receive inputs like sound or videos by adding such functions. If you take a look at things like prices, artificial analysis.ai provides various models for comparison. GPT-45 is now the best performing model.

I'm a model, and then there's the recent Cloud 3.5 Sonnet model. It's right next, and it's the paid version of the Pro version of Jemi AI 1.5. Here, the scores are the best, and next, among the previous models of Cloud, the large model, Oss, is now here. Next is the GPD 4 Mini. So, these various performances are said to be comparable to the old Direct PT 4 level of performance. Even though it's a much smaller model, and if you look at the speed, it's overwhelmingly the fastest. So personally, what I'm looking forward to is the speed. When you need a level of performance like GPT 45, you don't need it, but you need a very good performance, very fast, and if you want to process it cheaply, well, the most expensive model now is the old model of Cloud 3, the Office model of Lude 3. It's the best to buy, and next, now the price has dropped by half from GPT 4. So, this is.

We display the normal price for one million tokens at $7.5. For the new model like GPT-4 Mini, it is $0.3. Compared to GPT-45, it is 25 times cheaper, so you can consider it as such a model. When it comes to cost-effectiveness models, the Llama 38 model is usually the cheapest and relatively usable model, which is around 20 cents. On the other hand, for models like GPT-45 Mini, it's around 30 cents. So, if you take a look at the price trend so far, the newly released GPT-40 Mini is a cost-effective model. Depending on what criteria you use, input tokens are generally cheaper, while output tokens are a bit more expensive. It's measured to be around three to six times more expensive. Therefore, when generating with a ratio of around 80 to 20 for input and output, the price comes to about 24 cents. Looking at this price trend, back when GPT-3.5 was released,

Based on the standard, this is March 23, 2023. At that time, the price of one million tokens was around $2, so even back then, it was considered cheap. However, the price kept dropping. With the release of GPT 3.5, they continued to lower the price while improving performance little by little, which led to overwhelming competitiveness. The price dropped below $1.60, $1.20, $0.70, and even further. Despite the decrease in price, the performance is much better than back then. They have been making such decisions continuously. So, based on this announcement, as you mentioned, there is intense competition in the GPT model size, going in the opposite direction. Therefore, the competition for model sizes is intensifying.

We used to focus on making bigger models to see who can create a smarter model. But now, the trend is to see who can make it smaller while still making it smart enough. So, I saw a discussion on this topic here, and I strongly agree. Let me elaborate on my thoughts a bit more. There is an enormous amount of text on the internet and in the internet cloud, but this text is very messy. However, somehow, we have managed to gather a lot of this text and create what we call a language model. We created a language model, and with this, we have been using it. Initially, we started with smaller models and gradually made them larger and more efficient to compete. So now, instead of the current model, for example, with GPT-4, well, we usually...

Trillion parameters, this guy says there are 1.76 variables that can be adjusted, and there are also models like LAMA 3 villain model that briefly appeared, which usually include the model size in their names. This model was created by Meta Facebook. There are models like what we commonly refer to as cost-effective models, such as the 81-year model. So, to understand this size a bit, there is something called Chinchilla string. If you call it that, there is a paper. DeepMind, in March 2022, quite a long time ago, when this research was published. So, the research result is, when training this model to improve performance, what kind of balancing needs to be done. It's about how performance comes out when various operations are performed, how much performance comes out when various operations are performed, as well as the size of the parameters and token size. This is about how to train.

You can think of it as data, how much training data you provide, and what size model you choose, depending on these, the necessary calculations are almost organized like this. However, when considering these model sizes, calculations, and performance, balance is necessary. This is what they say. Here, what they are talking about is, there are training tokens in the training and there is a model size. So, naturally, if you increase both of these, the computer will go up because it needs to perform more calculations. When learning, a lot of calculations are needed, but depending on how you balance this, you can improve performance more efficiently without increasing computer calculations. This has empirically been shown through experience. So, for example, instead of just increasing the model size, at what point should you increase the model size?

There is a training token that can achieve optimal efficiency and performance at that time. Of course, it's better to put in a lot of training tokens, but rather than increasing them to a certain number, it's better to increase the number of training tokens and adjust the model size accordingly. It's better to slightly increase the model size rather than increasing it too much, and it's better to increase the training tokens to a certain extent and perform calculations. We have mainly developed in a way to balance like this. But these days, the trend is to make the model size smaller and instead provide a lot of training tokens for more learning. Also, if you provide a very good quality of training tokens, the performance improves. They are focusing more on that area. So, in the past,

To improve performance without using too much computational power, we are balancing these two aspects. Nowadays, we are reducing the size, increasing the training tokens, and enhancing the quality. By creating a large language model, we can generate high-quality training data. Using this, we develop a smaller model called Small LM, which performs well. Currently, the focus is on who can create better models. In cases like AraMa 3, we are moving towards training in a way that was traditionally considered appropriate.

People often express that even though a model like Bill has been trained much more than models with many more parameters, its performance is much better. Microsoft's Phi model also follows this concept. Models like GPT-4 Mini, which was released yesterday, belong to this category as well. The tweet mentioned earlier talks about how large language models, such as the large model, focus on memorization and reasoning abilities because they have an immense number of parameters.

Rather than creating an answer by thinking, it is about taking words from data with a lot of vocabulary and combining them appropriately to respond. In contrast, in the case of small models, only the thinking capacity remains, and because the number of parameters is now decreasing, only the thinking ability remains, and memorization is reduced. However, the reason why they perform well enough is that these small language models have sufficient thinking ability, so they can create good answers using things like chains of words. Let's take an example. For instance, if there is a known drug A and a known drug B, and if they are administered to a patient simultaneously, what will happen? When asking this question, in the case of models like this language model, it remembers the contents of these two drugs among the internet texts it has encountered in its numerous parameters. So, it remembers and produces the answer.

You connect the words you are memorizing like this to create an answer. But how do small language models do it? For example, if there are medicines A and B, what properties A medicine and B medicine have, and based on this, what actions occur, what changes happen, and ultimately, what will happen to the patient. This is how they create answers. Nowadays, language models use search or tools. So, they request information such as content or data about each drug from external databases, bring it, put it here, infer based on it because they are smart, and create answers. In this way, they can retrieve information from external databases and use it to make inferences about the properties of drugs, for example, the 3D structure of a drug.

With 3D structures, running simulations, deriving results, and creating based on them. Small language models are fast, affordable, and have sufficient reasoning abilities. Therefore, such applications are considered more suitable for tasks commonly referred to as agent tasks. Instead of large and heavy things, agents now perform tasks quickly. So, to answer this question, you need to plan ahead and go through these steps to predict how things will work and be created. Thus, it is natural for Cass to say that creating large models first is a matter of course, and then evolution naturally follows.

I am talking about evolving into a model that is both young and has excellent thinking abilities. This is a topic that I can really relate to, and I think those who are interested will find it intriguing as well. So, I will briefly introduce it and wrap up the video.

GPT-4o mini 소개, 작은 언어모델의 진화 (2024.7)

3D프린팅부터 게임까지 시도해본 것들을 기록으로 남깁니다.
배우고, 시도하고, 만들자

말러랩

한국어 기능이 대폭 향상되고 빠른 성능에 중점을 둔 가성비 높은 새로운 GPT 4 미니 모델이 출시되었습니다.

5 모델은 이제 최고 성능의 모델이며, GPT 3.

5 터보는 좋은 점수를 받으며 가성비 좋은 모델로 자리 잡았습니다.

GPT 4 Mini는 특히 텍스트 처리와 다중 모델 추론에서 뛰어난 성능과 속도로 주목받으며 상위 모델에 비해 더 합리적인 대안을 제시합니다.

GPT 모델의 100만 토큰 가격은 시간이 지남에 따라 GPT-3.

5의 경우 2달러에서 GPT-40 미니의 경우 최저 20센트까지 크게 하락했습니다.

이러한 급격한 가격 하락은 GPT 모델 크기 간의 경쟁을 심화시켰습니다.

더 크고 스마트한 모델을 만드는 것에서 더 작고 비용 효율적인 모델을 만드는 것으로 초점이 이동하면서 GPT 모델 시장의 경쟁이 치열해지는 추세로 이어졌습니다.

모델 크기와 트레이닝 토큰이 성능에 미치는 영향

성능 향상을 위해 모델 크기는 더 작아지고 학습 토큰은 더 많아지는 추세입니다.

효율적인 학습과 성능 향상을 위해서는 모델 크기와 학습 토큰의 균형을 맞추는 것이 중요합니다.

메타 Facebook의 연구에 따르면 최적의 효율성을 달성하기 위해 모델 크기와 학습 토큰을 조정할 필요가 있으며, 현재 성능 향상을 위해 고품질 학습 토큰을 우선시하는 추세입니다.

언어 모델에서 계산 능력과 성능의 균형 맞추기

언어 모델은 크기를 줄이고, 학습 토큰을 늘리고, 품질에 집중하는 방식으로 개선되고 있습니다.

이 접근 방식에는 고품질 학습 데이터를 생성하기 위해 대규모 언어 모델을 만든 다음 더 작은 고성능 모델을 개발하는 것이 포함됩니다.

이 논쟁은 더 나은 모델을 만들 수 있는 능력과 전통적인 학습 방법으로의 전환을 중심으로 전개되고 있습니다.

작은 언어 모델의 성능은 매개변수가 적음에도 불구하고 강력한 사고 능력과 암기력 감소에 기인합니다.

이러한 모델은 사고 능력과 단어 체인의 사용을 기반으로 좋은 답변을 생성할 수 있습니다.

약물 특성에서 추론할 때 언어 모델 사용

언어 모델은 외부 데이터베이스를 사용하여 약물 특성에 대한 정보를 수집한 다음 해당 데이터를 기반으로 추론을 수행합니다.

3D 구조를 활용하고 시뮬레이션을 실행하여 다양한 약물의 효과와 잠재적 결과를 예측할 수 있습니다.

소규모 언어 모델은 빠르고 저렴하며 충분한 추론 능력을 갖추고 있어 이 분야의 에이전트 업무에 적합합니다.