IBM의 그래닛 3.0 모델과 인스트럭트 랩의 혁신 사례

IBM의 Granite 3.0 모델은 다양한 애플리케이션을 위한 오픈 소스, 다용도, 강력한 언어 모델을 통해 엔터프라이즈 AI에 혁신을 가져왔습니다.

### IBM의 혁신적인 AI 개발: 그래니트 모델과 인스트럭트 랩
* 🗽 지난주 저는 뉴욕에 있는 IBM을 방문하여 **오픈 소스 Granite 모델과 Instruct Lab**을 비롯한 최신 AI 프로젝트를 선보였습니다.
* 💻 "오늘 출시된 Granite 3.0은 로컬에서 실행할 수 있는 오픈 소스 모델 제품군으로, **다양한 규모의 전문가를 포함한 기업 사용 사례**에 맞게 설계되었습니다."
* 📊 Granite 모델은 **기업별 데이터를 활용**할 수 있도록 구축되어 기업이 개인 데이터를 모델에 통합하여 분류 및 요약과 같은 다양한 작업의 성능을 향상시킴으로써 AI 역량을 강화할 수 있습니다.
* 🔍 IBM의 접근 방식은 **페이월이나 인증 뒤에 숨어 있는 데이터를 활용하는 문제**를 해결하여 기업이 모델 학습에 귀중한 데이터를 활용할 수 있도록 합니다.
* ⚙ 혁신적인 정렬 기법으로 설명되는 인스트럭트 랩을 사용하면 **원래의 학습을 덮어쓰지 않고도 외부 지식을 Granite 모델에 추가**할 수 있어 미세 조정과 검색 증강 생성 간의 균형을 맞출 수 있습니다.
* 📈 전문가 혼합 모델의 도입으로 IBM은 **더 적은 활성 매개변수로 고성능**을 위한 옵션을 제공하므로 리소스가 제한된 환경에 배포하는 데 적합합니다.
* 🔗 또한, 오픈 소스 솔루션에 대한 IBM의 노력은 **Red Hat과의 협업**과 **Granite 모델 기반의 새로운 코딩 어시스턴트 통합**을 통해 잘 드러납니다.
* 🔮 마지막으로, IBM은 양자 컴퓨팅을 적극적으로 연구하고 있으며, **향후 AI와 양자 기술 간의 잠재적인 교차점**을 암시하며 이 분야를 더욱 혁신할 수 있는 가능성을 제시하고 있습니다.

So last week, I was in New York to visit IBM. They invited me there to show off all of their new AI projects, and there are two that I want to tell you about today because they were especially interesting. Before we dive in, I want to thank IBM for inviting me to New York, showing me around their new office, which was awesome, and thank you to them for partnering with me on this video. And why I'm especially excited is because both of the things I'm going to show you right now are open-sourced by IBM. So first, let's talk about the granite models. Granite 3.0 was released today. It is an open-source family of models produced by IBM, and they are permissively licensed with Apache 2.0 licenses. It comes in a number of different sizes, including a mixture of experts. And after I tell you about these, I'm going to show you what instruct lab is because that's super interesting as well. It allows you to give the raw models a lot of additional knowledge, but it's not Rag, and it's not

quite fine-tuning either. It's a new technique that they have at least. All right, so let's start from Granite. It is Granite 3.0, the third version of their flagship open-source large language model. These are relatively smaller models that you can run locally, and they even have a very small model that can run on a device with really good performance. These are general-purpose models really built for enterprise use cases, but you can use them for anything. And what makes them special is they have this raw intelligence that you can then take and then supplement with external knowledge. Now, here's how they're thinking about it, and I agree completely. Most of the publicly available data on the internet has already been baked into all of the frontier models. There's essentially no more data on the internet that hasn't already been used. So, what do you do after that? Well, you have a few options. One, you can create synthetic data, which we haven't seen a really strong example of a frontier model having used synthetic data. You could also do a lot more with the existing

data you have, and that's kind of the approach that the 01 models took with test time compute, basically allowing the model to think. And then there's the third bucket, which is data that is either behind authentication, behind paywalls, or siloed in different enterprises. And that third bucket is what IBM is going for with the granite model plus instruct lab. And so, what does that actually mean? Well, first of all, you have data behind authentication and paywalls. So, Reddit has data, X has data, Meta has their Facebook data. All of this data requires you to be logged in, and it is owned by these companies. Then you have data that is owned by enterprise companies. So, let's say you're an insurance company. You have an immense amount of incredibly valuable data that is not public, that you want to use to train your own models. That is what Granite plus instruct lab is really able to unlock for you. So, you take this raw Granite 3.0 model, you take all of your enterprise private data, and then you put it into the Granite model. And then all

of a sudden, you have an extremely capable model that is very intelligent at your specific vertical. All right, so let's read a little bit about the granite model now. So, the granite models come in two different flavors: one is the general-purpose model, and then the other is the mixture of experts. So, as it says here, there's Granite 3.0 8B instruct and base, as well as Granite 3.0 2B instruct and base. Now, especially for the 2 billion parameter model, this can easily fit on any laptop, any phone, and will still perform really well. I'm very excited to hear more about what IBM has in store for their Edge compute strategy. Then, they have the small mixture of experts models. So, first, they have a 3 billion parameter version with 800 million active parameters, and then they have a 1 billion parameter version with 400 million active parameters. So, very, very performant for their size. These models have been trained by IBM to be good at a few things and specific, especially around Enterprise use cases. So,

the new granite 3.0 AP and 2B language models are designed as Workhorse models for Enterprise AI, delivering strong performance for tasks such as retrieval augmented generation (rag), classification, summarization, entity extraction, and Tool use. These compact, versatile models are designed to be fine-tuned with Enterprise data and seamlessly integrated across diverse business environments or workflows. But again, they're just raw models; you can use them for anything you want. Now, obviously, IBM, they cater to businesses, so that is what they built the models for and all the tooling. But you can easily use these models for your own personal use cases as well. They go on to talk about instruct lab, which I'll touch on after this, but instruct lab they describe as a revolutionary alignment technique. So, not quite fine-tuning, but it allows you to give this Granite Model a bunch of additional knowledge without overwriting any of the core knowledge it was first trained with. The other they released is the guardian family of models, which are basically alignment models, very similar to how meta has llama and Lam guard.

Granite has granite and granite Guardian, so although you might not want to use it for your personal use cases, if you're in a business environment, you definitely want to make sure that these models are aligned to your best practices and your guidelines. Let's take a look at some of the benchmarks. So, this is hugging fa's open llm leaderboard V2, and as we can see, compared to similarly sized models, there's gran 3.0 AP instruct, llama 3.1 AP instruct, and mraw 7B instruct. Granite 3.0 leads the pack. Now, here's a benchmark that shows how good it is at rag. So, as we can see here against those two other models that I already mentioned, with faithfulness, it is coming in at the top, and at correctness, it is also leading the pack. Here we have cybersecurity benchmarks, and of course, these are the benchmarks that matter if you're an enterprise and you want to use the granite models for your business. For the public benchmarks, again, Granite wins, and here are some examples of the benchmarks.

that are being tested SEC MML U cyber metric 80 cyber metric 500 LinkedIn sect trust assessment and so on. And of course, if you're in the Enterprise environment, what's also important tool calling. So let's look how it performs in tool calling against other models. Again, these same three models, Granite, Llama, and Mistral, all about the same size. Granite, you can see almost across the board, beats the other two models in tool calling. But they also released, as I mentioned, mixture of experts models. So I want to tell you a little bit about those, and I really like mixture of experts models because it allows you to have a larger model, but the actual active parameters that are being used when you're running inference are far fewer, making it much easier to run larger models on lower spec computers. So Granite 3.03 B, A 800m. And let me actually take a second to explain how to read this. So it's Granite 3.0, that's the model name. 3B is the total number of

parameters, and then the 800m means that there's 800 million active parameters being used in the mixture of experts. Here, they also have the granite 3.0 1B a400m delivering high inference efficiency with minimal trade-off and performance, trained on over 10 trillion tokens of data. The new granite Moe models are ideal for deployment in on-device applications, CPU servers, and situations requiring extremely low latency. Both of these mixture of experts models come open source, open weight, and you can download the Bas version or the instruction-tuned versions. I'll drop a link to the Hugging Face pages and these articles in the description below. Before I get to instruct lab, I just want to also mention IBM is releasing their own coding assistant powered by the granite model. They are really continuing to get deeper into coding assistance, and I absolutely love their embracing of open source. It could not be more clear with their acquisition of Red Hat just a few years ago. If you're not familiar with Red Hat, they

are open-source enterprise software solutions. They have a Linux distribution for enterprise environments. They have OpenShift, which is a Kubernetes-based container platform for building and deploying applications at scale. They have the Ansible automation platform. They have the OpenStack platform, which is for building and managing cloud computing environments. And then they have the middleware, again, all of which is open source. And here's another announcement that I didn't actually know was coming until I saw it today. So my favorite agentic framework, Crew AAI, is now integrated into the Watson X platform. So now you can build agentic workflows and deploy them in an enterprise environment on IBM's services. All right, now let me talk about Instruct Lab because I didn't actually understand it until they explained it to me at the event. So, as I mentioned, imagine you're an enterprise company. You take this Granite model, and it has the raw intelligence, but you want to fill it with your own enterprise data, private data, stuff that you don't want to put out in the public. So how do you do that?

well, typically, there were two ways: one, you use Rag, and Rag is great, but it's not always the most effective. Then, there's fine-tuning, but what I always understood fine-tuning for is more for guiding the tone and basically changing how the model responds, not really for giving it additional information. But now we have instruct lab by IBM, which is kind of this middle ground between those two. You're actually baking external knowledge into the model itself, but it's not quite fine-tuning; it's more like alignment, but giving it a bunch of additional data without replacing any of the original data. So, IBM and Red Hat's new open-source project is designed to lower the cost of fine-tuning large language models by allowing people to collaboratively add new knowledge and skills to any model, so not just a granite; this is any model. And instruct lab itself is open source, so I'll drop that in the description below. So, here's how it's described: it gives communities the tools to create and merge changes to LLMs without having to retrain the model.

from scratch instruct LAB Works by augmenting human-curated data with high-quality examples generated by an llm, lowering the cost of data creation. That generated data can then be used to customize or improve the base model without having to retrain it. So, it's a command-line interface that allows you to add and merge new alignment data to your Target Model through a GitHub workflow on your laptop. Lab's unique training regimen allows new information to be assimilated into the model during an alignment without causing the model to overwrite what it previously learned. So, basically, if you wanted to give a raw Model A bunch of new data, you would basically just use rag, but now we have this new technique where you can add a bunch of data without retraining the model from scratch, and that is what is unique about instruct lab. Again, it's open-source; you can contribute to it yourself. And there's one last thing I want to talk about, which is quantum computing. I don't know much about quantum computing, but IBM is deep in it.

they have their own quantum computer. They have already commercialized Quantum Computing, although it's very new still, but they have it, and they've been working on it for years and years. In fact, one of the lead researchers on the quantum team did a talk during the AI event and started talking about the combination of AI and Quantum Computing, which is interesting to say the least. I'm trying to get an interview with him, so hopefully I do, but let me know in the comments, do you want me to talk about quantum computer? I find it fascinating; I just don't know much about it yet. So, that's it for today. Be sure to check out the new granite models. Let me know if you want me to run them through my Benchmark in the comments below. Check out instruct lab; all of its open source. I'll drop all the links in the description below. And I want to thank IBM again for inviting me to their AI event, showing me all the cool stuff that they're building, and for partnering with me on this video. If you enjoyed this

video. Please consider giving a like and subscribe, and I'll see you in the next one.

IBM Unveils Granite 3.0 - Open Source Family of Small Models!

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Matthew Berman

오픈 소스 모델 제품군으로 출시된 IBM의 Granite 3.

0은 다양한 규모를 제공하며, 기업용으로 설계되어 로컬에서 실행할 수 있습니다.

이 모델은 기업 소유 데이터뿐만 아니라 인증 및 페이월 뒤에 있는 데이터를 대상으로 외부 지식으로 원시 인텔리전스를 보완하는 것을 목표로 합니다.

Granite와 결합된 Instruct Lab은 모델 학습을 위해 귀중한 개인 데이터를 활용할 수 있는 새로운 기술을 제공하여 기업을 위한 고유한 솔루션을 제공합니다.

Granite 모델은 일반용과 전문가용의 두 가지 유형이 있으며, 2B 모델은 모든 노트북이나 휴대폰에 장착할 수 있을 정도로 컴팩트합니다.

이 모델들은 엔터프라이즈 AI용으로 설계되어 검색 증강 생성, 분류, 요약과 같은 작업에서 강력한 성능을 제공합니다.

IBM의 Instruct Lab은 Granite 모델에 고유한 정렬 기술을 제공하며, Guardian 제품군은 비즈니스 모범 사례와 정렬을 보장합니다.

그래니트 모델의 성능과 전문가들의 조합

그래닛 모델은 라마 및 미스트랄 모델에 비해 도구 호출 기능이 뛰어납니다.

전문가용 모델을 혼합하여 제공하므로 더 적은 수의 활성 매개변수로 더 큰 모델 크기를 만들 수 있어 저사양 컴퓨터에 이상적입니다.

그라나이트 모델은 오픈 소스이며, IBM은 그라나이트 기반의 코딩 어시스턴트도 출시하고 상담원 워크플로우를 위해 왓슨 X 플랫폼에 Crew AI를 통합했습니다.

IBM의 인스트럭트 랩: 언어 모델 미세 조정을 위한 새로운 접근 방식

IBM의 Instruct Lab은 원본 데이터를 대체하지 않고 외부 지식을 모델에 베이킹하여 언어 모델에 Rag 사용과 미세 조정 사이의 중간 지점을 제공합니다.

이 오픈 소스 프로젝트는 모든 모델에 새로운 지식과 기술을 공동으로 추가할 수 있도록 하여 미세 조정 비용을 낮추는 것을 목표로 합니다.

Instruct Lab의 고유한 훈련 요법을 통해 모델의 이전 학습을 덮어쓰지 않고도 정렬 중에 새로운 정보를 흡수할 수 있습니다.

IBM은 양자 컴퓨팅을 성공적으로 상용화했으며 수년간 이를 위해 노력해 왔습니다.

양자 팀의 수석 연구원이 한 AI 행사에서 AI와 양자 컴퓨팅의 교차점에 대해 논의했습니다.

연사는 해당 연구원과 인터뷰를 요청하고 있으며 양자 컴퓨팅에 대해 더 자세히 논의할 의향이 있습니다.