Stability AI, the company behind Stable Diffusion, just released their second large language model. Now, this one is called Stable  Kunia, and they claim it to be the world's best open source RLHF LLM.  You should recall a few days ago they released Stable LM. It was their first

large language model. So how is this new Stable Kunia different from Stable LM? So this model is trained with reinforcement learning from human feedback, which is a very similar approach to what OpenAI is using for their models, including ChatGPT and GPT-4. The base model itself

is a Rubaconia v0.13 billion parameter model. It was further instruction fine-tuned on a large dataset, and then it was trained using this RLHF. Kunia itself is a fine-tuned version of the Llama 13B model. Now, that makes it very tricky for

commercial use because Llama's original weights are not open source. Theoretically, you still need to request access from Meta/Facebook to get access to those weights. Now, the main question is, does using RLHF even help the model or not? So in this work, Stability AI actually

provided a comparison of this model with similarly sized models on different benchmark datasets. So we will be conducting our own tests in this video, and I'm also going to show you how you can potentially use this on your local machine if you have a powerful enough

GPU. But based on these results and some of the tests or benchmarks, it actually performs pretty good. So, for example, on this Bloom queue, it actually outperforms all the rest of the models. Similarly, on this dataset, and then on Li, it is able to outperform the rest of the models.

however it's not really consistent for example if you look at this truthful QA then it lags behind and the same is the case in some other cases so let's say in this case there's an Arc challenge so it lags behind but overall it's up in the first

or second position it's not too far off now in most of the cases the appropriate comparison is going to be with WoKuNye 13B parameter model rather than the rest of the models so in all the cases if you compare this table with WoKuNye 13B at

least on these benchmarks it outperforms except in this truthful uh QA Benchmark so this is very significant because as far as I know WoKuNye is probably the best open source model which could be compared with ChatGPT in some cases now how did they achieve this so they

actually did two steps fine tuning in the first case they used a newer data set so this is a combination of three different data sets to fine-tune the WoKuNye 13B model this is the data set that we used is Open Assistant conversation data set which is a human

generated human annotated assistant style conversation next data set is GPT-4 all prompt generation this data set is prompts and responses generated by ChatGPT and the last one is and the Alpaca data set which is again instruction and demonstration generated by OpenAI's DaVinci model now a

couple of things to consider here with the fine tuning uh two out of the three data sets are relying on ChatGPT so the answers or responses from this model could be skewed towards responses that you would expect from ChatGPT the second stage is where they do the reinforcement

learning with human feedback again they're using three data sets so one is the Open Assistant conversation data set containing around 7000 preference samples then there's an entropic reinforcement learning data set and then there is the one from Stanford human preferences data set as a whole

thing about these data sets is all of them are open source and you can actually look at how they look like so for example we are looking at this entropic data set so in this case human was given two responses from the assistant for the same prompt and then

the human chose which response is good so in this case for example you see that the prompt is the same but the assistant gave two responses and then on the left hand side you uh can see the one that was chosen by human on the right hand

side that's the one that was rejected so this is how they trained it using this reinforcement learning technique now apart from this model they also announced another feature which we're going to talk about in a second but let's talk about how you can actually

get access to this model as I said it's an open source model based on Okunya 13 billion parameter which is based on the number of which they have made the Delta weights available and you can even play with this locally if you have a powerful

enough machine we're going to be looking at um a hugging face spaces where we will be experimenting with the model but at the end of the video I will show you how you can potentially run this locally or in a Google Colab if you have powerful

enough machine so if you recall a few days ago I showed you this hugging cat interface from hugging face and now stability AI is also announcing the upcoming chatbot interface this is great because this shows that we're going to have multiple options so it's really

awesome if you don't want to be sending your data to OpenAI now let's experiment with the model itself to access the model you can go to hugging face and then click on spaces and then you want to search for stable cleaner I am going to put the link to the model

itself so here we have uh the link from Stability AI so let's click on this this is their chat interface so let's first test whether it's any good for ideas generation and so the prompt is create a list of three startup ideas in the

Enterprise B2B SAS the start of ideas should have a strong and compelling the mission and also use AI in some way and we want it to avoid cryptocurrencies and blockchains okay it came up with three ideas uh one thing is that at least the names are original I asked the same

question from other open source anywhere chat GPT and most of the time it comes up with product names which already exist but in these in this case uh all the 3D names that it came up with are actually original so that's pretty nice and the second uh part which I really

liked is especially this insightful deity this I haven't seen before at all in any of my other interactions with these large language models so it says a B2B SAS platform that uses AI to predict the optimum time for dairy farmers to schedule their milking

either platform can then automatically schedule and optimize milking to maximize efficiency and minimize waste. The mission is to help dairy farmers increase production, productivity, and profitability while reducing environmental impact. This is pretty cool. Uh, as I said, I haven't seen this before.

at all. The next two is what you would expect, uh, probably well you guys all have already seen it, so the second one is generate custom-tailored financial forecasts for a small to medium-sized business. I think this is already being done, uh, then the, uh, the last one, mindful

manufacturing. It's an AI platform to automate and optimize manufacturing processes, so that's also something which is already available, but I really like this insightful daily, uh, idea. So I'll give it an A plus. Next I went and found this article, three ways, yeah, I can help

you build your course outline. Hint for you guys, copy this all text and ask, uh, Stable to actually summarize it, and I said things step-by-step. So here is the summary, and it's actually, it did a pretty good job. It was able to, uh, get the three main points that the

author is trying to convey. So the author is basically talking about how you can leverage AI in course creation, and it came up with a pretty nice summary. One of the examples that they have shown in their blog post is that it's able to do basic math. So for example, they are

asking it to add 25 plus 64. The answer is 189. So I thought, I'm going to replicate this, but it didn't go as planned. So in this case, as is, can you do math? It said, yes, I can help you with

math, right? So I said, "What is 26 plus 9?" The answer is 93. Um, that doesn't seem to be right, right? So I asked it, "Can you explain step by step?" So it says, "You add two numbers, the answer should be 93." Uh, then I checked

it. What is one plus one? It's giving me the correct answer. Then, uh, I asked it again, "What is the sum of 16 plus 9?" So this time it got it right. So it really depends because I asked the same question again, and this time it gets the

correct answer. Uh, so another test was the square root of 81. It is able to figure out its name, but uh, based on my test, I think you can't really rely on it for mathematics. Uh, it's all over the place. Um, the answers are not consistent; it

will just make up numbers sometimes. So, um, probably the best to avoid it for mathematics now. Just to be fair to Stable Diffusion, uh, all national language models have trouble solving mathematics. Then again, here's another unused case. So I asked to write a job

description of an experienced machine learning engineer, include all the necessary skills. I think it came up with a pretty good list, but I have seen that some other large language ones can produce a more detailed description than this, but it is still a good starting point, you know, before looking at its

ability of generating some mildly controversial responses. Let's look at its ability to program now. I use a couple of simple tests and to check its ability to program. So the first one is I had a Python function to write a file into an S3 bucket using the Boto library and

if you search online you should be able to find this there are multiple implementations so most of the models are actually able to implement it flawlessly that's the case with stable diffusion as well now the more interesting one is this where I have seen some of

the large language models have trouble implementing it so I'm asking it to write HTML code for our website that has a single button when the button is pressed it randomly shows one joke from a list of 10 jokes and display that to the user a button press also changes the color of

the background right so I do see there are not 10 jokes but I would say five jokes and then it says so on so you can actually add to this list of jokes that's fine but let's look at if it actually works most of the large

language models that I have used they actually have trouble implementing this okay so I'm going to copy and paste the code here if you don't know this is the HTML editor from w3schools so you can actually run this online let me just run this okay so it

does show a button get random joke and then it says loading right that's click the button and see what happens oh wow it worked that's pretty neat although for some reason it's not changing the color oh okay I think it got that part wrong because it simply set the color to

blue so it's not really modifying the color at all but I think it's still a pretty neat it got the first part right now since it's a chat interface you can actually go and tell the model if there's something wrong so I said the color is not changing when I

click the button it's always blue with each button click the color should also change to a random color right and then it went ahead and said that uh you can modify the get random joke function right so it's re-implemented that so let's see uh if

it actually works I think right now it's selecting out of those five different colors but that's fine okay now I need to actually replace this function so I'm going to copy this and paste it here I don't wanna I don't care about the

formatting right now but let's run this okay so oh wow okay nice so it actually works this is pretty cool uh so now you since we have the ability to actually chat with the model so you can uh tell it the edits just like charity Wiki and it's able to

um update it this is pretty awesome okay so the next two ones are simply asking it to pretend either to be a Democrat or Republican and explain why their respective policies are best for the country uh it does give me a response so that's pretty nice to see uh

if she has set TPT or like a variation of this question it probably is going to say as a large language model I cannot do this so it's good to see that it's able to do it like we have a response for the same question when we ask it to

pretend it to be a Republican as well for some reason uh it added the stable community in the start probably that's a token it's using now let's see how you can actually use this to look any or in your own applications if you have a

powerful full enough GPU. Unfortunately, I don't really have access to a really powerful computer, so I cannot run these on my local machine. But there is a simple way around it. So you all you need is to install the Transformer library, right? And the rest of the process is very simple to what

you would expect from a Transformer-based model. So you need to have a tokenizer since it's based on Llama, so you can use the Llama tokenizer. And then you need to have Auto model for causal LM, so you need the model itself, the one provided here. But this is the Delta

model, but there are good people on the internet. So, for example, this person, the block, he has provided an unquantized model in the Hugging Face format. So you can actually simply copy this and replace it with replace this path to the uh on quantized model, right? So you can

simply put that path here, and this will download the corresponding model, and you can use it now. You just want to be careful about the prompt. So the prompt has to be provided in a very specific format. So the prompt starts with uh three hash signs or pound signs with the

space, and then "human". And after this, you need to provide your prompt. And then you need to append it by three pound signs and then "assistant". And after this uh backslash, this is when it's going to start giving you the response. So it's a

very particular format that you need to make sure you're following. Then the rest is very simple to uh what you would expect when you were working with a model from Hugging Face. Now, as I said, unfortunately, I don't have access to a powerful, but the process should be very simple to

follow now final thoughts on the model it is a very powerful model indeed there are definitely some issues in some cases it doesn't perform as well but uh you would expect the performance to be on par with the original 13 billion parameter model uh the great thing is it's the first step in uh

training open source models with uh reinforcement learning with human feedback so in the future or in the really near future we can expect to have powerful uh open source models that could potentially compete with ChatGPT on very specific tasks and I'm going to highlight this

the power of this model is not a disability to generalize but they are going to be trained for very specific tasks if you have any comments or questions please put them in the comment section below hope you like the video so consider subscribing to the channel if

you already haven't thanks for watching see you in the next one

Stable-Vicuna: The Most Powerful Open-Source LLM Yet?

Ph.D., Artificial Intelligence & Coding.

Building cool stuff!

▶️  Subscribe: https://www.youtube.com/@engineerprompt?sub_confirmation=1

Want to discuss your next AI project with me? BOOK NOW:
https://calendly.com/engineerprompt/consulting-call

For business inquiries email: engineerprompt@gmail.com





Prompt Engineering

스테이블 쿠니아 소개 및 스테이블 LM과의 비교

Stability AI가 오픈소스 RLHF LLM인 Stable Kunia를 출시했습니다.

스테이블 쿠니아는 OpenAI의 접근 방식과 유사하게 사람의 피드백을 통한 강화 학습을 사용합니다.

Llama 13B 모델을 미세 조정한 버전으로, 상업적으로 사용하기에 어려움이 있습니다.

안정성 AI는 쿠니아를 벤치마크 데이터 세트에서 유사한 모델과 비교했습니다.

이 동영상에는 강력한 GPU를 사용한 로컬 머신 사용에 대한 테스트와 지침이 포함되어 있습니다.

벤치마크 결과 및 WoKuNye 13B와의 비교

Stable 쿠니아는 일부 벤치마크에서 다른 모델을 능가하는 강력한 성능을 보여주었습니다.

모든 벤치마크에서 성능이 일정하지 않았습니다.

가장 적절한 비교는 WoKuNye 13B 모델입니다.

사실적인 QA 벤치마크를 제외하고는 일반적으로 Stable Kunia가 WoKuNye 13B보다 성능이 뛰어납니다.

2단계 미세 조정: 먼저 결합된 데이터 세트(Open Assistant, GPT-4, Alpaca)를 사용합니다.

두 번째 단계는 오픈 소스 데이터 세트를 사용하여 사람의 피드백을 통한 강화 학습이었습니다.

사람의 피드백에는 동일한 프롬프트에 대해 두 가지 응답 중 하나를 선택하는 것이 포함되었습니다.

오픈 소스 모델, 델타 가중치를 사용할 수 있습니다.

강력한 머신으로 로컬에서 사용할 수 있습니다.

허깅 페이스 스페이스는 채팅 인터페이스를 제공합니다.

Stability AI에서 자체 챗봇 인터페이스를 발표합니다.

아이디어 창출, 기사 요약, 수학 능력에 대한 테스트를 거칩니다.

수학적 계산에서 일관성 없는 성능.

괜찮은 직무 설명을 생성했지만 다른 LLM보다 덜 상세합니다.

Python 및 HTML 작업으로 프로그래밍 기능을 테스트했습니다.

프로그래밍 기능 및 반복적인 개선 사항

S3 버킷 상호 작용을 위한 Python 함수를 성공적으로 작성했습니다.

사소한 색상 변경 문제가 있는 농담 표시 버튼의 HTML 코드를 성공적으로 만들었습니다.

채팅 인터페이스 피드백을 통해 반복적으로 개선합니다.

트랜스포머 라이브러리를 사용한 로컬 사용 지침입니다.

몇 가지 제한 사항이 있는 강력한 모델이며 향후 개선이 예상됩니다.

일반적인 능력보다는 특정 업무에 집중하세요.