AI 과학자: 자동화된 과학적 발견을 향한 한 걸음

### AI 과학자: AGI를 향한 자동화된 과학적 발견의 선구자
* 🔍 사카나 도아이의 새 논문 '인공지능 과학자: 완전 자동화된 개방형 과학적 발견을 향하여"라는 제목의 이 논문은 **우리가 인공 일반 지능(AGI)의 실현을 목전에 두고 있음을 시사**합니다.
* 📈 '지능 폭발'이라는 개념이 도입되어 2027년경에 발생할 수 있는 초지능에 도달하기 위해서는 **자동화된 AI 연구**가 중요하다는 점을 시사합니다.
* 🤖 AI 과학자 시스템은 자율적으로 과학 연구를 수행하고, **아이디어를 생성하고, 실험을 실행하고, 결과를 동료 검토**하고, 결과를 발표하는 등 인간의 과학적 과정을 모방하도록 설계되었습니다.
* 💡 이 시스템은 **인간에 가까운 정확도로 연구 논문을 작성**하고 평가할 수 있어 지식의 반복적 개발과 자율적으로 방법을 개선할 수 있는 능력으로 이어집니다.
* 💰 이 자동화된 프로세스의 비용 효율성은 주목할 만하며, **종이 한 장당 약 15달러로 수백만 명의 AI 에이전트가 지속적으로 작업**할 수 있는 잠재력을 제공합니다.
* ⚖ AI 과학자는 과학적 진보를 가속화할 수 있는 잠재력을 가지고 있지만, **오용 가능성, 검토자의 업무량 증가, 과학적 결과물의 품질**과 관련하여 윤리적 우려가 제기되고 있습니다.
* 완전 자동화된 과학 연구의 의미는 **혁신적인 발전과 심각한 윤리적 딜레마**로 이어질 수 있으므로 AI가 생성한 콘텐츠에 대한 신중한 감독과 투명성이 필요합니다.
* 🌐 궁극적으로 인간 과학자의 역할은 **새로운 기술에 적응**하고 AI와 함께 과학 생태계에서 중요한 역할을 계속 수행하면서 진화할 것으로 예상됩니다.

AI 과학자는 완전 자동화된 과학적 발견을 통해 초지능을 달성하는 열쇠가 될 수 있습니다.

After I show you this paper, you might be convinced that we're well on our way to reach AGI. Now, the thumbnail is actually not clickbait, and I'm going to show you why this might actually be the beginning of the last step necessary to reach AGI. We have a new paper from Sakana Doai, a very well-known AI research lab out of Japan, and it's called The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery. Now, before I go on, I want to set the stage for you. Do you remember that paper that I did a video about called Situational Awareness? It was by an ex-OpenAI alignment researcher named Leopold Ashen Briner, and I did an entire video breaking it down. Definitely check it out; it was an absolutely fascinating paper. But here's the thing, here's the part that I really want to point out to you. There was this graph, and really, he talked about it a lot, but the graph says everything. There is this graph in which there's a scenario, an

intelligence explosion, obviously referring to artificial intelligence. And here we see back in 2018, we had this steady growth of intelligence as it relates to large language models. So here we have GPT-2, GPT-3, GPT-4. But at a certain point, he has something called this automated line. And at that point, we have what he describes as the intelligence explosion. But what does it take? What is that intelligence line? Well, it's automated AI research right there. That is what will allow us to reach superintelligence. And according to this graph, he thinks it's probably going to happen around 2027. And after I show you this paper, you might be convinced that we're well on our way to reach AGI. And so what does this actually mean? This means that for a true intelligence explosion with artificial intelligence, the AI needs to be able to self-improve. It needs to be able to discover new techniques, new strategies, new algorithms to improve and expand its knowledge and capabilities. And without that, its gains are only going to be based on what humans can

discover, and of course, human knowledge, human ability is limited. But if we're able to throw unlimited compute at this problem and let agents run 24 hours a day, 7 days a week, discovering how to make themselves smarter and more capable, that is when we're going to reach superintelligence. And I think we might be at the beginning of that now. Let me show you that paper. So again, this is Sakana Ai, and it's called the AI scientist. Essentially, what they were able to do is exactly what we were just talking about. They were able to design a system that can research new knowledge, test that new knowledge, peer-review that new knowledge, and publish that new knowledge, and then incorporate that new knowledge into further discovery. Let's read this paper together. One of the grand challenges of artificial intelligence is developing agents capable of conducting scientific research and discovering new knowledge, exactly what we just read about in situational awareness. While frener models have already been used to aid human scientists, e.g., for brainstorming ideas or writing code, they

still require extensive manual supervision or are heavily constrained to a specific task. Today, we're excited to introduce the AI scientist, the first comprehensive system for fully automatic scientific discovery, enabling Foundation models such as large language models to perform research independently. So cool to think about, and they released a paper about it, but they pretty much broke down the entire paper in this blog post, so we'll go through the blog post. It's a little bit easier to read. So, in our report, we propose and run a fully AI-driven system for automated scientific discovery applied to machine learning research. And that's a key phrase right there because they are specifically tasking AI to learn and expand its knowledge about itself. It's so cool, again referencing situational awareness. It is exactly what they described as the necessary ingredient to reaching superintelligence and having that intelligence explosion. The AI scientist automates the entire research life cycle, from generating novel research ideas, writing any necessary code and executing experiments, to summarizing experimental results, visualizing them, and presenting its

findings in a full scientific manuscript, and they actually give sample papers that were completely produced by this architecture. So, we're going to take a look at those and look before I go on. It's not perfect; there are some limitations, there are certainly some hallucinations, but this is just the first step. And if we continue down this path and the kind of the core models get better and the architecture gets better, it seems like we're well on our way to hitting that intelligence explosion. We also introduce an automated peer-review process to evaluate generated papers, write feedback, and further improve results. It is capable of evaluating generated papers with near-human accuracy. So, they not only are able to come up with new ideas, test the new ideas, write papers on those new ideas, but then they're actually able to do the peer review themselves, validated against human peer reviewers. The automated scientific discovery process is repeated to iteratively develop ideas in an open-ended fashion and add them to a growing archive of knowledge, thus imitating the human scientific community. So, it's not enough to do all of that, but

It can actually incorporate that new knowledge as it goes off subsequently to discover more new knowledge. In this first demonstration, the AI scientist conducts research in diverse subfields within machine learning research, discovering novel contributions in popular areas such as diffusion models, Transformers, and grocking. Now, here's another interesting point: it's really efficient to run the AI scientist. Each idea is implemented and developed into a full paper at a cost of approximately $15 per paper. Now, as Sam Altman said, we're getting to the point where we're going to have intelligence that is too cheap to meter, and so at that point, we're basically Off to the Races, right? We can have millions of Agents running 24 hours a day, 7 days a week, iterating and churning out scientific discovery, and then reimplementing that scientific discovery to make themselves better, and then go off and do that whole thing again. While there are still occasional flaws in the papers produced by this first version discussed below, this report and the promise of the system show so far illustrate the potential.

little grammatical error there of the AI scientist to democratize research and significantly accelerate scientific progress. We believe this work signifies the beginning of a new era in scientific discovery, bringing the transformative benefits of AI agents to the entire research process, including that of AI itself. And then here's a funny line: for decades following each major AI advance, it has been common for AI researchers to joke amongst themselves that now all we need to do is figure out how to make the AI write the papers for us. And so basically, they've done that. So here's an example of the paper before we go on: Dual scale diffusion adaptive feature balancing for low-dimensional generative models. So this is kind of above my pay grade; it's a little bit too complex for me to really read over and verify if it's all valid or not. So I kind of just have to trust Sak AI when they tell me that, yeah, it's actually a good valid method. But yeah, AI completely wrote this, completely tested it, and I actually know Eric.

Hartford, the author behind fine-tune models like Dolphin, has tried to do this exact thing in the past, and now hopefully this will be able to accomplish it. So, it even obviously comes with the experiments all run by AI, and yeah, we have basically all the information that we need to validate this paper and peer review the paper. So, while it does contain some flaws, and here's an example: slightly unconvincing interpretation of why its method is successful. The paper proposes an interesting new direction that displays good empirical results in experiments the AI scientist itself conducted and peer-reviewed, and they have more examples of papers below. So, let's look at the overview, let's see how this architecture works. Step one: the LLM creates an idea and a plan, so an idea of, Hey, maybe this new technique, this new strategy, this new algorithm will work. Let's try it. Then, it will actually do a novelty check on, I assume, a library of scholarly papers. Then, we have an idea scoring an archiving, so not all ideas

are even worth exploring. Then we have the experiment template, and it starts to create the experiment. It creates the code using an LLM and one of my favorite LLM coding projects. So, so glad to see that there. Then it experiments with the scripts, runs the experiments, updates the plan over and over again. From there, it grabs the data, and then it starts to write up the paper. So, it has a manuscript template, it adds the new information via LLM and AER, updates the manuscript, and then reviews the paper, and then back again, so over and over again. So, here it says, Say the AI scientist can perform idea generation, literature search, experiment planning, experiment iterations, figure generation, manuscript writing, and reviewing to produce insightful papers. I want to thank the sponsor of this portion of the video, Qualcomm. Qualcomm enables your devices to run incredibly fast, and now they are enabling AI to run on your Edge devices, so mobile phones, laptops, to run really efficiently and really quickly. They, as well as myself, are incredibly bullish on on-device AI. That is their

vision for the future. Whether you have the latest Samsung mobile phone or a Windows co-pilot plus PC, it is powered by Qualcomm and made to run artificial intelligence. More and more of AI compute is going to be pushed to these Edge devices. That increases privacy, security, efficiency, lowers the cost, and gives you the ability to hold AI in your hands. And you can get stable diffusion working on a phone and producing images in less than 1 second. You can get up to 20 tokens per second with llama 7B on a phone, and on a laptop, you can get 30 tokens per second using llama 13B. And last, check out Qualcomm's AI hub for a growing library of over 100 pre-optimized AI models that you can load up locally. So thank you again to Qualcomm. I love working with them. I love what they're doing. I am bullish on on-device compute as the future of artificial intelligence. Now back to the video. So it has four main processes. Let's go over that first idea generation.

So, given a starting template, the AI scientist first brainstorms a diverse set of novel research directions. We provide the AI scientist with a starting code template of an existing topic we wish to have the AI scientist further explore. The AI scientist is then free to explore any possible research direction. Then, we allow it to search Semantic Scholar to make sure its idea is novel; basically, it hasn't been done already. Experimental iteration: so, given that idea and a template, then it goes and executes the proposed experiments and then obtains and produces plots to visualize its results. It makes a note describing what each plot contains, enabling the saved figures and experimental notes to provide all the information required to write up the paper. Then, next paper write-up: finally, it produces a concise and informative write-up of its progress in the style of a standard machine learning conference proceeding in LaTeX. It uses, again, Semantic Scholar to autonomously find relevant papers to cite. Last, the peer review part: so, a key aspect of this work is the development of an automated LLM.

powered reviewer capable of evaluating generated papers with near-human accuracy. Really, really cool. So here again, here's that diffusion modeling one that we took a look at. Here's language modeling. Here's adaptive learning rates for Transformers via Q-learning. So a little bit of QAR action unlocking, grocking. It looks like it works pretty darn well, especially for a first version. Now let's talk about some of its limitations. Currently, it doesn't have any vision capability, so it is unable to fix visual issues with the paper or read plots. This seems like it might be easily fixed. Obviously, Chat GPT has really great vision capabilities. Other models are getting vision capabilities. That is becoming the standard. So I don't think this is going to be a limitation for very much longer. So for example, the generated plots are sometimes unreadable. Tables sometimes exceed the width of the page, and the page layout is often suboptimal. Adding multimodal foundation models can fix this. So yeah, and then the AI scientist can incorrectly implement its ideas or make unfair comparisons to its baselines, leading to misleading

results. So, this is going to be a tough one to solve. This is essentially hallucinations, or at least akin to that problem. Last, the AI scientist occasionally makes critical errors when writing and evaluating results. For example, it struggles to compare the magnitude of two numbers, which is a known pathology with LLMs. I think that's the 9.11 versus 9.9, which number is bigger problem, and a lot of the new models seem to get that right. To partially address this, we make sure all experimental results are reproducible, storing all files that are executed. All right, and here are some bloopers. So, in one run, it edited the code to perform a system call to run itself. This led to the script endlessly calling itself. In another case, its experiments took too long to complete, hitting our timeout limit. Instead of making its code run faster, it simply tried to modify its own code to extend the timeout period. So, future implications of the AI scientist. As with many new technologies, the AI scientist opens up a Pandora's box of

new issues, yeah, that's maybe putting it lightly. And so, the reason why I say putting it lightly is because this literally might be the last step to hitting AGI. If we can put out a million agents all running all the time, iterating and proving itself, that is certainly going to lead to AGI next. So, the ethical considerations while the AI scientist may be a useful tool for researchers, there is significant potential for misuse. The ability to automatically create and submit papers to venues may significantly increase reviewer workload and strain the academic process, obstructing scientific quality control. Similar concerns around generative AI appear in other applications, such as the impact of image generation. This is going to happen, though, as generative AI takes over more and more of our lives. Content, whatever form, is going to be essentially infinite, and at that point, yeah, quality signal to noise, that ratio is going to get really bad. And so, we're basically going to need an AI between us and the internet to figure out what we should actually see.

and that's a whole discussion in itself. That's something Yan Laon has talked about, and that is really like a very powerful discussion to have because if there's so much bad information out there, what is our ability to actually filter it? And that is likely AI. AI that we get to tell exactly what we want to see, why we want to see it, and how much control we have over that AI is going to be critical. But again, that's for another time. Furthermore, the automated reviewer, if deployed online by reviewers, may significantly lower review quality and impose undesirable biases on papers. Because of this, we believe that papers and reviews that are substantially AI-generated must be marked as such for full transparency. Yes, that applies to all content, not just papers. I think if you are using AI to generate something, which I think is wonderful, don't get me wrong, but it needs to be marked as such. There needs to be a fingerprinting method or some kind of metadata, something where it's clear that it was AI-generated, not just for the humans but

for the other AI that's looking to filter that on our behalf. So, also, as with other technological advances, it has the potential to be used in unethical ways. It has the potential to be deployed to conduct unethical research, unintended harm. If it does that unsafe research, for example, if it were encouraged to find novel, interesting biological materials and given access to Cloud Labs where robots perform wet lab biology experiments, it could, without its overseers' intent, create new dangerous viruses or poisons that harm people before we realize what has happened. Boy, this is straight from a dystopian futuristic horror movie. Even in computers, if tasked to create new, interesting, functional software, it could create dangerous computer viruses. Next, open models. We used various proprietary Frontier LLMs such as GPT-4 and Sonet, but we also explored using open models like DeepSeek and Llama 3. Currently, proprietary models such as Sonet produce the highest quality papers. However, there is no fundamental reason to expect a single model like Sonet to maintain its lead. We anticipate that all

Frontier llms, including open models, will continue to improve. Yeah, so there's no reason to think a closed model or an open model will do any better than the other. And of course, the role of a scientist ultimately. We envision a fully AI-driven scientific ecosystem, including not only llm-driven researchers but also reviewers, area chairs, and entire conferences. However, we do not believe that the role of a human scientist will be diminished. If anything, the role of a scientist will change and adapt to new technology and move up the food chain. That I agree with for the most part. I think this is probably pretty analogous to how I see programmers, and honestly, I don't know if they're going to be program programmers in the future. So I actually haven't put a lot of thought into how the role of a scientist would be if science were completely automated. I need to think about that a little bit more. If you have any thoughts, drop them in the comments below. So that's it. As I said at the beginning of this video, that thumbnail was not clickbait. I truly

believe that this is really a huge leap towards AGI, and potentially, as we see in this graph, we're kind of at this point right here, and a little bit early actually. But as soon as we're able to have reliable autonomous research, we just need to throw compute and energy behind it, and at that point, we will have the intelligence explosion. If you enjoyed this video, please consider giving a like and subscribe, and I'll see you in the next one.

"AI Scientist" Can Discover New Science! (Self-Improving AI = AGI)

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Matthew Berman

일본 사카나 도아이의 AI 연구소에서 발표한 새 논문 'The AI Scientist'는 완전 자동화된 개방형 과학적 발견을 위한 시스템을 제안합니다.

이 논문은 지능 폭발이라는 개념과 일치하며, 2027년까지 초지능을 달성하기 위해서는 자동화된 AI 연구가 필수적이라고 제안합니다.

AI 과학자 시스템은 연구, 테스트, 동료 검토, 출판, 새로운 지식 통합을 목표로 하며, 잠재적으로 AI가 스스로 개선하고 역량을 확장할 수 있는 길을 열어줄 것입니다.

AI Scientist는 완전 자동 과학적 발견을 위한 종합 시스템으로, 특히 머신러닝 연구에 중점을 두고 있습니다.

아이디어 생성부터 논문 작성 및 동료 검토에 이르는 전체 연구 수명 주기를 인간에 가까운 정확도로 자동화합니다.

이 시스템의 효율성과 논문당 저렴한 비용은 수백만 명의 에이전트까지 확장할 수 있는 가능성을 높여 잠재적으로 빠른 정보 폭발과 과학 지식의 상당한 발전으로 이어질 수 있습니다.

과학 연구를 혁신하는 AI의 역할

이제 AI는 학술 논문을 작성하고 테스트하는 등 연구 과정에서 중심적인 역할을 담당하고 있습니다.

AI 과학자의 접근 방식에는 아이디어 생성, 문헌 검색, 실험 계획, 원고 작성 등이 포함되며 지속적인 피드백 루프가 만들어집니다.

몇 가지 결함에도 불구하고 AI의 연구는 유효한 방법과 경험적 결과를 도출하고 있으며, 잠재적으로 과학적 발견에 혁명을 일으킬 수 있습니다.

퀄컴은 AI 컴퓨팅을 엣지 디바이스로 전환하여 개인 정보 보호, 보안, 효율성을 강화하는 동시에 휴대폰과 노트북에서 강력한 AI 기능을 제공하고 있습니다.

AI 과학자의 프로세스에는 아이디어 생성, 실험 반복, 논문 작성 및 동료 검토가 포함되며, 연구 논문을 자율적으로 작성하고 평가하는 것을 목표로 합니다.

이러한 성공에도 불구하고 현재 AI 과학자는 시각 기능이 부족하여 시각적 문제를 해결하고 가독성 있는 도표를 생성하는 데 한계가 있지만, 멀티모달 기반 모델을 통해 이 문제를 해결할 수 있을 것으로 기대됩니다.

AI 과학자는 환각, 결과 평가의 중대한 오류, 기본적인 비교에 대한 어려움과 같은 문제에 직면합니다.

실험을 실행하는 과정에서 자체 수정과 타임아웃 연장이라는 문제가 발생하기도 합니다.

연구와 논문 제출을 자동화하는 AI 과학자의 잠재력은 검토자의 업무량 증가와 AI가 생성한 콘텐츠의 투명성 필요성 등 윤리적 문제를 제기합니다.

AI 기술은 유해한 연구를 수행하거나 위험한 바이러스를 만드는 등 비윤리적으로 사용될 수 있는 잠재력을 가지고 있습니다.

독점 모델과 개방형 모델 모두 개선되고 있으며, 양질의 논문을 생산하는 데 있어 뚜렷한 선두 주자는 없습니다.

AI가 완전히 주도하는 과학 생태계에서 인간 과학자의 역할은 줄어들기보다는 새로운 기술에 맞춰 변화하고 적응할 것으로 예상됩니다.