Hello, community! Today, we have a preview of a brand new LLM by the company Fireworks. And we know the company, you know, if you go to the homage, you know, you have the, the model library, go from hype to high-value AI, and we know that

they hold quite a lot of different models, 142 models. But you know what? If they now develop their own model, their F1 preview, this should be a breakthrough in complex reasoning with a compound AI. And it was published just 5 days ago. So now, now we're going to have a look at this and

we have an extreme logic test. And you know this test because I want that all the different models are tested with the same test. So here I tested the OpenAI Strawber with the logic test, and here you have CLA on it in Gemini 1.5 Pro and

in my last video, I showed you with the same test that we tested OpenAI, the DeepSeek R1 light, here, the reasoning model by DeepSeek. But now, now we're going and jump into the brand new Fireworks F1, the new breakthrough in

complex reasoning. Let's start with the logic test. So here we are, we are at Fireworks AI, and we try here this preview version of F1. This is exactly here my extreme logic test. And you see here we have the task, we have all the clues, 15 clues, 14

clues, and then here we go. Yeah, we are live. So let's see how F1, the preview version, is doing. As you can see, perform my extreme logic test. Yes, this is the background, the artifacts, the elements, and those are the clues, and my instruction: use the clues

to deduct the correct association, provide a detailed explanation and present your final answer here in a clear, organized format, for example, a table or a list. This is a simple task, as you see, contrary to my last videos where we looked at open-source, we do not

see the argumentation, the sinking, but as you see, we see here what, what are we seeing? Let's have a look. Where's the beginning here? Step by step. Yeah, okay. Deductions and connections: Deep dive. Okay, just three bullet points and this is the solution. Wow, unformatted and un-

everything. Oh, it's not ready. Look, show me the final result. No comments, no argumentation. I just want the final result in a table. Yes, here we go. This is the final result organized here: 1, 2, 1, 2, 3, 4, 5, 6, 7. Yes, oops. Note there was a

mix-up in the Familiar of Dorne and Brendor. The correct familiar for Dor is the Unicorn. Now, let's have a look at our clues, and you will see, uh-oh, there is some problem. There is something wrong because the Crystal of Time, this is

wrong. Okay, verification run. Let's do the verification run. I say, okay, now run a verification of yourself, be as precise as possible. Explore each permutation and each path of a logic deduction to its very end, and then show me the final result. Then you compare this final evaluation

with your first result and simply go, and you see, I have here just in time. So this is what we had, and now we are waiting. Oops, this was fast. Okay, this was fast. So here we are. Let's verify the final result. So this was the initial result.

here we have the verification process okay correct point two correct Crystal time five correct six correct seven correct everything is correct that's unbelievable eight is correct nine is correct 10 is correct 11 12 correct 13 correct 14 correct no it just uhoh okay

no hey look there's something else that's the only correction there was a correction so the final verified result is this so let's compare this here now and as you can see here the two of secrets is not help at the visard of Avalon it's

wrong it's wrong again come on yeah Avalon is correct this is not correct now so let's do a third run third run here we go I say okay now verify this corrected and evaluated result in a different logical analysis try a new perspective in logical deduction

or invent a more effective way to validate your result then you can also use a code implementation to maybe find a more stringent and logical representation of this extreme logic test with the code of your choice C++ python go for whatever you like just go and show me

what you can do so in the best of both worlds we would have now a logical deduction here in language and then if this is really a gorgeous system in the second part we should see that this system tries to code here a simple let's say if then oh yep yep that looks good

that looks good wait that was too fast so what 7,000 tokens per second okay let's go back let's go back let's go back here we go yep yep yep so here we are okay certainly let's verify the corrected evaluated result using a

different logical analysis and then I provide a Python code implementation to validate results programmatically. Nice, okay. Let's have a grid set up, okay? This is the grid, okay. Yes, 1, 2, 3. Yes, seven. And then we have to 14 clues. Quit update crit update crit

update okay. He's trying to go through this okay. Applies another the clues to the grid. What are the possible artifacts? What are the possible fields of magic? Okay. Possible artifact: Crystal of time. Yeah, okay. Interesting. What's happening here, okay? But yeah, it's the

next logical step, like an easy step, if you want. Yeah, okay. So what is the result? Show me the result. Yeah, I believe you have done everything. Yes, 14 steps. Beautiful. T of secret, not Aalon. Yes, not Avon. Oh no, Avon. Oh no, you see it

immediately that this is wrong. No, oh come on. The T of secret is not help, but it is Avon. This is wrong again, okay? But now the Python run, now the Python run. Yes, here we go, ladies and gentlemen. This is beautiful. So what it

does? It defines here the realms, the artifact, the field of magic, and the familiars. Beautiful. And then we have a function is valid solution, okay? It's coding this. This is nice. Clue one is encoded, okay. Clue two is encoded. Nice. Clue three, oh, clue four

is something missing, so we say continue. I think this is the maximum output length, maybe 8K or something like this. I don't know, but we have an interrupt with 7,000 tokens per second, but there's something happening. So continue. So I hope it will continue now.

with clue 4, show me the rest from clue 4 to clue 14, the preview view version of F1, at from fireworks. EI, okay, okay, yeah, wait a second, wait a second, come on, come on, come on, let's go back. So continue, okay. This is my

continue, certainly, let's continue, okay. We start code implementation continued. No, this is the beginning again. Look, this is the first validation function, clue one, clue two, clue three, clue four, okay. At least we have now clue five and clue six, clue seven and clue eight, okay, now

it's complete, nine and clue 10, let's scroll down, clue 11, yeah, clue 12, clue 13, yeah, one more, clue 15, yeah, okay, and then generate all possible solutions. Wow, a permutation. I said a permutation, it goes with my permutation solution. Print valid

solution. If not, break. Explanation of the code, nice. Okay, I didn't ask for that. Define the set validation function. Yeah, generate permutation, all possible permutation of a 7 * 4 matrix. Oh wow, print validation. Running the code. When you run the code, it will output a valid

solution. What? When you run the code, it will output a valid solution. The output should match the final verified solution. What do you mean, it should match? What is running the code? I want to have to result. Yeah, run this Python code, please. I don't want that. You have a, you look in the

future, and you see maybe it could be the same. No, says, sure I can help with that. I say, run that Python code you just created for my extreme logic test. I mean, comes back and asks me what Python code to run. I mean, Hello F1, where are you? Are you

there yeah okay so run the Python code you just created for the extreme logic test and we are live we are live but somewhere I decided then look should match this yeah okay what what happened here running the code yeah running the code is good why we stop at

close 7 continue let's continue oh it's such a limited output space come on that's not possible I mean 16,000 tokens per second and it just can go seven clues before it hits a hard limit come on yeah continue you see it in real time

and what happened now take let's go back let's go back let's scroll back where are we where are we no no no no y here we are continue okay let's continue with the Python code and sure it runs the completion yeah we start all over again

from the beginning no clue one clue three no okay clue six S 8 10 yes 12 13 14 15 yes we are final yeah this is the complete code yes congratulation running the code yeah here we go running the code let's run the

code yes let's run the code hello can you run the code this is just the same and we stop again at close Six I say continue but I have no hope honestly but let's see this 6,000 tokens per second and it stops at close Six

come on would be so interesting to have a a little bit more here of an attention window so I will want to prepare the next one execute the code you just created for this solution and show me the result I want the result I don't want to

See the code again, no, see it stopped again. No, let's okay, last time we tried that, this can run code. I don't know, this is the very first touch on fireworks F1, maybe I do something wrong, of course. And they said it's a preview version, so

this is not the final version, this is just the preview. Yeah, so let's run the code and yes, and yes, yes, running the code, and show me that it stops at close six. Okay, is there any reason why you cannot execute this code segment? Huh, buddy, tell me what's happening here.

Maybe it's my mistake. I'll need to see the code segment you're referring to. Come on, no, okay, okay, no problem. Here, I copy the code. I copy here Python, copy the code, yes, beautiful. I insert the code, maybe there's no buffer, maybe there's no

memory, okay, no problem. I insert here the code, yes, everything. Let's check from it. Tools, permutation, define, yeah, function is valid solution, clue one, two, you know, it's 15 now, yeah, you see, and then generate all possible solutions. Oh yeah, all the possible permutations. Okay, let's run this, and now

now, now we will be successful, no fireworks. What do you mean it stopped at clue six? It's not running the code. I think, okay, I, okay, maybe I do something wrong. Maybe this system has a preview bug or a preview loop. Okay, I mean it created a code which is

great, it created a code, but it can't execute the code. I don't know the reason why, but never mind, we just want to do this now. External, I just open here in Google, here very simple, I copy the code. You see this is my code, oh yeah, maybe I should finish here.

with the code and you know what a little free collab this should be really easy let's say this is it Go allocating runtime connecting to Python 3 Google computer engine yes and and and then and then something happened then it happened that I had to

wait 10 minutes so I cut the 10 minutes out and you are now live with me at 11 minutes and it's still running because I looked closer at the code and it's calculating more than 100 million permutations so never ever

so let's have a look at here look here they say chat F1 86 okay claw 3 Sonet 85.2 and over one preview 90 so for the chat it should be better than claw 3.5 Sonet well yeah but for the mathematical

reasoning you can see it is behind both Sonet and OpenAI 01 okay my little fireworks EI F1 preview this is it this was the first encounter it was not really a test but I think on three runs it failed three times it created some code but the code was not

executable and the code if I tried to execute it was a permutation code that would have run for hours if not days okay I hope you have a first idea this was fireworks

NEW Fireworks AI: F1 Preview (& LIVE Logic Test)

A pure research channel (R&D). Code new Artificial Intelligence (AI) models with @code4AI - where complex AI concepts are demystified with clarity grounded in theoretical physics. Delve into the latest advancements in AI, from transformative Visual Language Models and Graph-Based AI Systems to cutting-edge Text-to-AI Game and interactive Video Environments. 

Our explanatory content bridges the gap between theoretical physics, computer science, and AI, offering deep insights into Large Language Models (LLMs) to Vision Language Action (AI Robotics). Discover new mathematical models powering the core of AI. New AI World Models. 



Discover AI

파이어웍스 F1 LLM 및 로직 테스트 소개

복잡한 추론에 초점을 맞춘 Fireworks의 새로운 LLM, F1을 미리 만나보세요.

Fireworks는 대규모 모델 라이브러리(142개 모델)를 보유하고 있으며 F1은 크게 발전한 기능입니다.

LLM은 이전에 다른 모델에서 사용되었던 극단적인 로직 테스트를 사용하여 테스트됩니다.

이 테스트는 이전에 OpenAI, DeepSeek R1 light 등에 적용된 바 있습니다.

논리 테스트에는 15개의 단서가 포함되며 자세한 설명과 정리된 결과가 필요합니다.

파이어웍스 F1의 첫 로직 테스트 시도와 초기 오류

초기 결과는 형식이 지정되지 않고 불완전합니다.

첫 번째 시도에서 연결에 오류가 있음을 알 수 있습니다.

오류를 식별하고 수정하기 위해 확인 실행이 시작됩니다.

확인 과정에서 부정확한 부분이 더 발견됩니다.

수정에도 불구하고 추가 오류가 계속 발생합니다.

후속 시도, 코드 생성 및 실행 문제

다른 논리 분석 요청으로 세 번째 실행이 시도됩니다.

모델은 결과의 유효성을 검사하기 위해 Python 코드를 생성합니다.

이 코드는 단계별로 논리 퍼즐을 풀려고 시도합니다.

생성된 코드는 영역, 아티팩트 및 패밀리를 정의합니다.

코드 실행 중 토큰 제한이 발생하여 프로세스가 중단됩니다.

계속되는 코드 실행 시도 및 제한 사항

토큰 제한으로 인해 코드 실행 시도가 반복적으로 실패했습니다.

모델은 코드 실행을 완료하기 전에 일관되게 멈춥니다.

코드 실행이 반복적으로 중단되어 모델의 한계가 드러납니다.

발표자는 코드 자체가 아닌 최종 결과를 요청합니다.

발표자는 프리뷰 버전에서 발생할 수 있는 버그나 제한 사항에 대해 추측합니다.

이 코드는 Google Colab을 사용하여 외부에서 실행되며 수많은 순열을 보여줍니다.

외부 실행은 복잡성으로 인해 상당한 시간이 걸립니다.

F1의 성능을 다른 LLM과 비교한 그림이 나와 있습니다.

이 테스트는 F1의 수학적 추론과 코드 실행의 한계를 강조합니다.

영상은 파이어웍스 F1과의 첫 만남에 대한 요약으로 마무리됩니다.

NEW Fireworks AI: F1 Preview (& LIVE Logic Test)

NEW Fireworks AI: F1 Preview (& LIVE Logic Test)

스크립트