so I have this idea for a business, and the thing that I'm lacking the most is time. So what I decided to do was put together a presentation showing you exactly what this idea is, why I think it's valuable, and then hopefully one of you builds it. And I actually think this could be a startup in itself, so hopefully you like it. Let's get into it. So I'm calling it Abstract AI. Obviously, call it whatever you want, but I think it's kind of a cool name because I think of it as an abstraction layer on top of large language models. So first, what is the problem? The problem is AI developers are under-optimized. And who am I actually talking about? Well, I'm talking about two groups of people mainly. One is AI product companies, so companies that are building and selling products or services that are powered by AI or existing products and services that are being enhanced by AI. And then the second group are large organizations that are looking to implement artificial intelligence internally at their company. Both of these groups of people consume a

lot of large language model tokens through API calls, but for the most part, they're taking a pretty naive approach to it. They're simply going to usually open AI's endpoint and choosing GPT 40, but there's a ton of room for optimization within that. That's really what this is all about. So the problems that they're facing are they're overpaying for Frontier llms. Typically, AI developers are not rolling their own models, and a lot of the time, they're not even hosting Frontier open-source models. They're usually just paying open AI. That is why open AI has the vast majority of Revenue right now in the AI World. Also, AI developers, because they're only building on one cloud service and they really only have access to one set of models. So let's say GPT models within the open AI ecosystem. They have a ton of platform risk if open aai decides one day to change their policy to change their model to charge more. They are exposed to all of these potential risks. Also, when you're going to a Frontier

Model, and that's the only model you're using. That tends to be the slowest and highest latency model out there. And here's the thing, for 90 to 95% of prompts and use cases, the Frontier Model is not necessary and is overkill. It's the most expensive and the slowest. They're typically not using a lot of algorithmic techniques that we've been seeing over the last months. So, Route LLM, Chain of Thought, Mixture of Agents, all of these algorithmic unlocks are just simply not being used. And all of this leads to really inefficient LLM usage. And so, that is what I believe Abstract AI can solve. So, Abstract AI will lower latency, lower cost, and offer more flexibility. And we're at this point where LLMs are essentially commodities. There are a ton of amazing offerings. Now we have a completely open-source, open-weight Frontier Model from Meta Llama 3.1 405b. And that is really just leveling the playing field for large language models. And in that sense, maybe

cost is not the priority, but when you're talking about potentially using billions of tokens per month, when you get into agentic systems, and you're deploying AI to companies with tens of thousands of people, cost actually probably still does matter. And if you're able to reduce cost by 20, 30, 80% in some cases, that's actually really meaningful. But here's the thing: developers generally don't really care which large language model is being used if they can get the same quality, the same consistency, and the same speed from some noname model compared to ChatGPT, they probably don't care. What they care about is consistency of response, quality of response, and at scale cost. So as I mentioned, most developers are using a single endpoint and a single large language model. They're not using any of the algorithmic unlocks that have been coming out, which will only add quality. Even if you do nothing else, if you simply use some of these techniques Chain of Thought mixture of Agents, you literally just layer that on top of whatever large

language model you're already using. You will get much greater quality, but these things take a lot of work to implement. They're not that simple, and the techniques are constantly evolving. There's new techniques coming out all the time, and so being up to date on the latest algorithmic unlocks is a lot of work for a developer and something that is not really their core expertise. So what is abstract AI in the most simple terms? Abstract AI is a single drop-in API replacement for whatever large language model you're using today. It connects to multiple large language models, everything from closed-source frontier to open-source to even local models, and it is the cheapest, fastest, highest quality responses on a per-prompt basis. So it's able to use at the very front of the stack something like Route LLM to determine on a prompt-by-prompt basis which large language model should be used, which algorithm should be used by determining which will be fastest, cheapest, and highest quality. And as we've seen from the Route LLM research paper, you can achieve 90% of GPT-40 quality for 80% reduction.

in the cost simply by routing the prompt to the right large language model. And again, I want to stress this includes locals. So, if you're deploying AI within your business and you have 10,000 employees, you can install a very small model on every single one of the computers. And with Route LLM wrapped around it, it will answer essentially for free 90% of the prompts locally. And that increases safety and security, which we'll get to in a moment. And then for anything else, for anything more sophisticated, for coding use cases, for anything that is more complex, that is when it outsources to a larger model, to a private cloud model, or even a closed-source cloud provider like OpenAI. And if this sounds familiar, it is essentially what Apple Intelligence is doing. Even though they're completely closed-source, Apple Intelligence is installing a small model on the phone that can work on your behalf. And then for any world knowledge or any more complex use cases, it outsources either to their own private cloud but a massive

model or eventually with permission to open AI. So, how does this all work? One, it uses Route LLM as it says right here to pick the right large language model for each prompt. So, Route LLM out of the box comes with a really nice machine learning algorithm that can determine which model a prompt should go to. But that model, that ML model, can actually be trained for your specific use case and for your plethora of different models that you want to use, and even include algorithmic techniques to determine what quality is going to come out of each of those models. Then you layer on some of these algorithmic unlocks that we've talked about, either on the open-source smaller models or even on top of the larger Frontier models, although it's kind of less important at that level. You optimize on the small models using an agentic framework. So, Crew AI, for example, mixture of Agents Chain of Thought, and you can really just tell it what you want to get out of it and why. And so, if speed is what you're optimizing for, you

can actually tell the API about that, and then it will change its routing to optimize for speed. If the highest quality is what you're looking for based on your prompt, it will do that. So you can kind of manually tweak it as you're going as well. Then, as mentioned, you outsource to a frontier model only when required, kind of like what Apple intelligence does. So then, what, as I mentioned at the beginning, consistency and quality are kind of the two most important factors when an AI developer is deciding which large language model to build on top of. So, Abstract AI will have built-in benchmarking to optimize the response consistency, and the benchmarking can be custom. So, for a specific use case within a company or within a product or service, you can actually have benchmarks that you're running against for any new large language model that you want to include or any algorithm that you want to put on top of that. Then, it should also have eventually built-in caching. So, one tried and true way to basically guarantee consistency, improve quality, decrease cost, and latency is by using caching.

autogen does this really well. So if you submit the same prompt again and again, there's no reason why you actually need to hit a large language model again and again. You are expecting the same response, so simply return the cached version. This will optimize speed, cost, and consistency. So then, what is the broader vision of this project? I think there is a bunch of different areas for expansion, but one that makes a lot of sense to me is prompt management. Now, because Abstract AI is being inserted into such a critical path for any company, for any AI Builder, you all of a sudden have this really great expansion path as well. And one direction is upstream towards the user, and you can start doing prompt management, whether that's User permissioning, Group permissioning, Company rules, and then also versioning. So you can have different versions of prompts, and again, all of this tied into the benchmarks which we already talked about. And then from there, again, because this product is being inserted into the most critical area of an AI developer's workflow, you

can really expand in a bunch of different ways. So a bit rough, but hopefully you enjoyed this. I really think there is a great product, if not an actual business, to be built from this. And I don't have time, but if you decide to build it, reach out to me. I'd love to talk to you about it, and I'd love to help you in any way I can. If you enjoyed this video, please consider giving a like and subscribe, and I'll see you in the next one.

This Could Be a MASSIVE AI Business...and Now It's Yours 🤑

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Matthew Berman

Abstract AI는 AI 개발자들이 대규모 언어 모델을 더 효율적으로 사용하도록 돕는 API 대체 솔루션으로, 비용 절감, 지연 시간 감소, 유연성 향상을 목표로 한다.

이 솔루션은 Route LLM을 활용하여 프롬프트에 따라 최적의 모델을 선택하고, 다양한 알고리즘 기술을 적용하여 응답의 품질과 일관성을 높인다.

궁극적으로 Abstract AI는 프롬프트 관리 기능을 확장하여 사용자 권한, 버전 관리 등을 통해 AI 개발 워크플로우를 개선하는 것을 목표로 한다.

💡 추상적 AI 소개: LLM 최적화 솔루션

Matthew Berman은 시간 부족으로 직접 구축하지 못하는 사업 아이디어, 즉 **대규모 언어 모델(LLM)을 추상화하는 레이어인 "추상적 AI"를 소개합니다**. 이 아이디어는 AI 개발자, 특히 AI 제품 회사와 AI를 내부적으로 구현하려는 대규모 조직의 최적화되지 않은 상태를 해결하는 데 중점을 둡니다.

💸 주요 문제점: 과도한 비용과 제한적인 유연성

AI 개발자는 일반적으로 **개방형 AI의 GPT-4와 같은 프론티어 LLM에 과도한 비용을 지불하고 있습니다**. 또한 단일 클라우드 서비스와 모델 세트에 의존하여 정책 변경 또는 가격 인상과 같은 플랫폼 위험에 노출됩니다.

**대부분의 프롬프트와 사용 사례에서 프론티어 모델은 불필요하게 과도합니다**. 개발자들은 Route LLM, Chain of Thought, Mixture of Agents와 같은 알고리즘 기술을 충분히 활용하지 못하고 있으며, 이는 비효율적인 LLM 사용으로 이어집니다.

✅ 추상적 AI의 솔루션: 비용 절감, 빠른 속도, 유연성 향상

추상적 AI는 **단일 드롭인 API 교체**를 통해 이러한 문제를 해결합니다. 다양한 LLM(폐쇄형, 오픈 소스, 로컬 모델)에 연결하여 각 프롬프트에 대해 가장 저렴하고 빠르며 고품질의 응답을 제공합니다.

Route LLM을 사용하여 **프롬프트별로 가장 적합한 LLM과 알고리즘을 결정하여 비용을 최대 80%까지 절감**할 수 있습니다. 또한 로컬 모델을 활용하여 안전성과 보안을 강화할 수 있습니다.

⚙️ 작동 방식: Route LLM, 알고리즘 잠금 해제, 벤치마킹

Route LLM은 **각 프롬프트에 가장 적합한 모델을 선택하는 머신 러닝 알고리즘을 사용합니다**. 이 알고리즘은 특정 사용 사례에 맞게 훈련할 수 있으며, 알고리즘 기술을 레이어링하여 품질을 향상시킬 수 있습니다.

일관성과 품질을 보장하기 위해 추상적 AI는 **응답 일관성을 최적화하는 내장 벤치마킹 기능을 제공합니다**. 벤치마킹은 사용자 정의가 가능하며, 캐싱을 통해 속도, 비용, 일관성을 더욱 최적화할 수 있습니다.

🚀 확장 비전: 프롬프트 관리 및 그 이상

추상적 AI는 **프롬프트 관리로 확장하여 사용자 권한, 그룹 권한, 회사 규칙, 버전 관리 등을 제공**할 수 있습니다. 이 제품은 AI 개발자의 워크플로에서 중요한 위치를 차지하므로 다양한 방식으로 확장할 수 있습니다.

🤝 결론: 협력 및 추가 지원 요청

Berman은 이 아이디어가 **실제 사업으로 발전할 가능성이 있다고 믿으며**, 구축에 관심 있는 사람들에게 연락을 요청합니다. 그는 도움을 제공하고 싶어하며, 비디오가 유용했다면 좋아요와 구독을 부탁합니다.

아이디어 소개: Abstract AI - LLM 추상화 레이어

시간 부족으로 아이디어를 공유하고, AI 개발자들의 비효율성을 해결하고자 함.

AI 제품 회사와 AI를 내부적으로 구현하려는 대규모 조직이 주요 대상.

AI 개발자들이 Frontier LLM을 과도하게 사용하고, 단일 클라우드 서비스에 의존하여 플랫폼 위험에 노출됨.

Abstract AI는 지연 시간 감소, 비용 절감, 유연성 향상을 목표로 함.

Abstract AI의 핵심 기능 및 작동 방식

단일 API로 여러 LLM에 연결하여 가장 저렴하고 빠르며 품질 좋은 응답 제공.

Route LLM을 사용하여 프롬프트별로 최적의 LLM 및 알고리즘 선택.

로컬 모델을 활용하여 비용 절감 및 보안 강화 (Apple Intelligence와 유사).

Route LLM을 통해 각 프롬프트에 적합한 LLM을 선택하고, 알고리즘을 통해 품질을 향상시킴.

속도 또는 품질에 따라 라우팅을 조정 가능.

응답 일관성을 최적화하기 위한 벤치마킹 및 캐싱 기능 제공.

확장 가능한 비전: 프롬프트 관리

사용자 권한, 그룹 권한, 회사 규칙 및 버전 관리를 포함한 프롬프트 관리 기능 확장 가능.

아이디어를 구현할 사람에게 연락을 요청하며, 도움을 제공할 의향을 밝힘.

This Could Be a MASSIVE AI Business...and Now It's Yours 🤑

This Could Be a MASSIVE AI Business...and Now It's Yours 🤑

스크립트