Hello community! Yes, we have a new idea, and it is a modular system that we built on purpose. And I know what you're going to say: 'Hey, I have my proprietary AI system, let's say, one from OpenAI, and I'm happy. It does everything I want.'

Just pay 200 bucks a month or 2,000 bucks a month, and it's working great. But imagine you would understand AI; imagine you would be interested in coding AI yourself; imagine you would be interested in building a specific AI application for your particular task in

your university, in your research lab, in your company. This is now what we're going to talk about in this video: how to build a modular, open-source AI system. And there's a particular reason why we talk about this, and the first part is here: organic solar cells and testing

all different available molecular configurations. And you have your two typical molecule clusters. A brand new publication by UC Berkeley, Stanford University, and Microsoft Research, December 10th, 2024, and they are talking here about a missing link to making the development of LLM systems an engineering discipline.

And you know what they do? They think, 'Hey, we have decades of software engineering, and if we look here at the complete dynamic of those and we compare it now to AI,' you know what we found? They found that the key to building now reliable and modular AI

systems is specification. So this means before you build anything, you have to have a precise description of the expected behavior of your AI, of the inputs that are available, and the output of each component if you build it in a modular way. So you design an AI system for one

This is a specific job; it's not like one open system can do everything for 4 billion people, for all their tasks. No, we have small, local, high-performance expert systems that we have to build for our specific tasks, and we are

laser-focused on this. And yes, it could also be a small language model. Now, in the development of new organic solar cell molecular clusters, there are a lot of machine learning methodologies for property prediction that we hope machine learning will help us with. The instrument, the tool that we use is a

graph neural network for molecular property prediction. However, as you know, maybe with conventional GNNs and even the commercial GNNs that we can buy, these systems struggle with higher-order structures. This now includes molecular features such as aromatic rings that we encounter in the

development of organic solar cells. So there's now a team—and I'll show you the publication in 5 minutes—developing a new, dedicated AI system only for developing organic solar cell molecules. So they built a highly dedicated, laser-focused graph transformer

network, and its only job is to analyze all the possible structural patterns at both the atomic level and the ring level within our target molecules. This is it, and you know what? Their very first try, their zeroth try—they expected maybe we could reach the middle class of our competitors—they already achieved a 22% improvement over the best competitor in the class. So whenever you build dedicated, targeted AI systems, they can be small, they can be open source, they can live on your local computer infrastructure, but they can only do one task.

But in this task, they are almost unbeatable, and this is also kind of the statement that we see here from UC Berkeley and Stanford. Yeah, they say if you want to build such a reliable AI system, you have to have a clear view on the

specification: what is the expected behavior of the system? And it cannot be everything for everybody in the world. And we do have challenging, and unfortunately today we have these monolithic systems, which constrain the development of these AI models only to a few companies: OpenAI, Microsoft, Google, and DeepMind, who have the money to invest billions of dollars. But this monolithic nature of these AI models also makes it difficult to address issues when these models produce incorrect outputs. We, as a community, can do nothing; we have to wait until we can pay

for the next model, and this is not okay. So therefore, we want to be able to build our own systems. Berkeley tells us, you know what? It is simple. If we look at a complete software development cycle, it is just there was here this new

methodology: if you decompose a complex system into simpler ones and you reuse existing components in the system, this lowers the barrier to entry for building complex AI systems, and this makes it so much easier because you

have your building blocks, and you know that the building blocks work; you just have to combine them in a particular way, and then you can run your optimization. Now, building modular systems is closely related to building a more reliable AI system in total because these are often built as a

model system by combining smaller, already tested, already verified, and thereby reliable EI components. We do not start with an open, untested black box environment. So great, let's look at the specification to optimize the performance of our OSC.

molecules, and I'm not an expert in organic chemistry; I'm a theoretical physicist, but I know something, so I share my knowledge with you. Key properties for OSCs are, of course, these six properties: power conversion efficiency, highest occupied molecular orbital—if you think in the old, really classical model—

here, band gap energy, differences between the levels, maximum voltage available for solar cells, current density, and so on. So we have the classic properties, and then we have data sets, and this is the beauty: We have, for example, from Howard the Clean Energy Project database here, with

2.3 million organic semiconductor molecules; detailed descriptions—we know everything about them. Then we have the "Hard Organic Photovoltaic" data set from 2016 with 350 small molecules and polymers, here also with the complete structure. Then we have different data

sets here where we have the donor polymer molecules, including the parameters that we want to use for the calculations. We have further data sets, and you get it. Now we have to ask, "Hey, how do we combine this? What do we want to achieve? What is our job to be done?"

And then we build our own AI system. Now, if you're new to AI, it's a little bit of a threshold because we need an understanding of what a neural network is. If you are not familiar, never mind; in February 2019, you should read this paper here by

MIT about graph neural networks, and they also show you in this particular paper here graph isomorphism networks, or GIN networks as we call them. I also have a video here on the graph isomorphism problem that we encounter here, and one particular test that is really important

is the Weisfeiler-Lehman test here. And if you're a little bit better in mathematics, I have a second video for you on new topological layers on graph neural networks with filtration and persistent homology, and a new graph layer for it. It is great, and also a second

Just imagine February 2020: we already had a new paper here by Stanford University and Harvard University. Stanford here had the strategies for pre-training graph neural networks, and they already applied GNNs—the GIN architecture—on molecular property prediction. So you see it took just months

to get this idea and apply it directly here in materials science, in chemistry, organic chemistry. And we want to build your molecular predictions. It is rather easy: you have an input graph like we always have, and then you have some center node and you have some context anchor node, then you have your

graph neural network. You define a specific neighborhood—one-hop, two-hop, or three-hop neighborhoods—and you also have masking in the process, like we know it here for example from the BERT, from classical BERT systems. And then we have simply here the task to predict specific molecular

properties, given here the training data set and the understanding and the content of the training data set. Yeah, there's now not only the GIN network, but now there is an extended network where we integrate the expressive message-passing graph neural network frameworks by taking here also the edge attributes into

The message passing, and we get here some better results, but this is just a side effect. If you want to see how we code this, in this video I show you. This is 2 years old, the message passing in the Deep Graph Library implementation. And also, if you are not familiar with how we

built here a graph machine learning representation, a computational graph, given we have some graph structure data with a tensor structure. I explain here how we do this in graph stage and so on, here in this particular video. So there are resources for you to learn immediately, within hours, up to 100%.

So now we are back to the main task. We want to build our new AI, our model AI system for some efficient OC property prediction, and we want to respect this new ID by UC Berkeley and Stanford University concerning the modularity of our approach. So we understand the model failed

up until now because we have this complex aromatic ring, see, in a molecular compound. And now we want to build a model that takes specific care about this complexity that we encounter in our simulation, in our prediction. So therefore, we have to understand it first, as I told you, as a theoretical physicist, so

I'm just simple-minded here. We have different ring structures in our molecular compounds. We have the aromatic rings that we find here, but we also have heteroaromatic rings here with sulfur and nitrogen or oxygen inserted. And then we have fused ring systems, so multiple

rings sharing different atoms, forming larger conjugated systems. So you have all kinds of mix-ups. But what we want to do is, now we have here the classical flat atomic-level graph neural network module, captures the local chemical structures like in the

Functional groups that we encounter. And now we build here a specific focus: we build now our ring-level cross-attention. We do this here with a Transformer module. This is known—this is known code—and this now focuses here on modeling the global relationships within the ring system, so

we are no longer restricted here to the immediate local neighborhood (our epsilon), but we can now go global on a molecular level. Great! So, of course, if we have both systems, we have to bring them together. We do this with an inter-level GNN network module

that facilitates interaction between the atomic and the ring representations. And then we want to build a layer; therefore, we merge all the complete information together. At the end of each layer, we have a fusion. So we go from a GNN to a

Transformer, then we have here the inter-level GNN module, and we have here a fusion layer that brings everything together. This layer architecture allows us to stack multiple layers, given your computational hardware. Great! So we have an idea—you see we start here with what we have, then we

add here—if we understand that the molecular characteristics of our solar cell molecule are determined by the ring structure—we build a specific part in our AI system that focuses only on this ring structure and the interaction of those ring structures with the other elements

in our, if you want, simulation. There's another software package, PyTorch Geometric, that I show you here in this video. And you know, I just checked that GPT for Omniverse. So we operate here in a very simplified code. I just wanted to make sure, hey, can

ChatGPT4 Omni code: If we tell it, 'I want this, and this, and this,' can you code it?  GPT, on the first run, not bad at all. So it starts here with the atom-level GNN process. Here, the atom-level graph uses standard GNNs to encode

here the local atomic structures and their interdependencies. Each node (each atom) receives messages from its neighbors and updates its specific embedding. So we have here, from torch geometric, import our Gin convolution. So we define it here, and we have your symbol forward pass; you see it's

as easy as this with our code editors. It's no longer complex to code this because these libraries are available on GitHub. Every system should know this now. If you want challenges, it's in how to design the system. If you can decide or describe how you want to

design a system, the code implementation is done by the code editors. Great! The ring-level cross-attention: As I told you, here we use a transformer-based cross-attention mechanism to encode the relationships between the rings. We have both the node embeddings and the

edge attributes here for a better representation of the complexity. We could add theoretical or virtual nodes (I'll explain this in a minute). So we have here the class of the ring-level cross-attention. Here, beautiful! You see, you don't have to go with 01 or O1 Pro, or

if you go with Sonet or whatever you have as your code system—even old-fashioned Fortran Omni can do this—and then we bring it together in the inter-level message passing. To exchange information between the atoms and the ring structure, we use a bipartite graph structure. Each

Atoms can send messages to their corresponding rings and vice versa.  The ring itself can also communicate. We can interconnect different rings and different classes of rings; we can make it really complex. But the simplest code, which I'll show you in the instructions, is here.

You have the class defined here, and of course, our hierarchical fusion operation. We're designing a layer for this self-designed AI system. The researchers—I'll show you in a moment—are calling this the 'ring former'.

We already have multiple ring formers, so it's not a unique label or term, but never mind. We combine the atomic level, the ring level, and the interlevel module into a single layer, so we can stack multiple layers.  You see, Omni is

doing this for us. Of course, it's just skeleton code; it's not a perfect code implementation, but you get the idea. Now let's come to the prediction, because we built this system so that the AI predicts exactly what we want, so we don't have to do this experimentally.

Given a particular molecular graph—a flat graph—the objective is to learn a property prediction model that predicts the target property values, *y*.  *y* can be any of the six properties I showed you: power conversion efficiency rate or molecular orbital...

We already talked about the importance of the ring system, that it's characterized by different types of rings and their interconnections.  Great limitation, yep. Since I see this from the perspective of an AI physicist, just to make sure we're both on the same page.

So you might ask why we're looking here at the ring system. Why are they so important for solar cell efficiency? Now, the electronic and structural properties of those organic semiconductor molecules depend heavily on the ring system. So we have alternating single and double bonds in aromatic or heteroaromatic rings, and they create, in the classical picture, delocalized pi electron systems.

The rings, with extended conjugation, absorb light much more effectively, enabling a broader spectrum of sunlight to be captured by our organic compound system. We can modify them to have better charge transport, and what's really nice is that these aromatic rings often add chemical and thermal stability to the complete molecule we're designing, making our organic semiconductor material much more durable in real-world testing. Great! Now we have everything we...

know. We know how to implement it; we saw some code. So here we have the final idea: We start out with an atomic-level graph. We construct on top of this a ring-level graph, and then we build an explicit interlevel graph to describe all the...

complete connectivity on a local and global level. We build a hierarchical graph system, and we know how to handle graph transformations and graph neural networks, in particular, the isometric forms. Now, building these particular layers is easy peasy, but I want to...

show you the official publication. They do it exactly like we did. We start with an atomic-level graph. At the GNN, we have the message passing, great! Then we build the ring connections; we have the ring-level graph (if you want) with our attention...

mechanism from the Transformer, the cross-attention mechanism, and then we build here the inter-level message passing again with a GNN. So you see exactly this, and we form here the first layer. Now, as I told you, you can have multiple layers; you see here in the original publication

here you have a second layer, and then you just pull together everything, and this is it. And this output defines now your specific OC properties. And here is now the main documentation today: RingFormer, a ring enhanced graph transformer for gaining solar cell property prediction by the

Hong Kong Polytechnic University. Some beautiful, beautiful ideas! Please note we have the code available here at GitHub. You have here the RingFormer code if you want to experiment with this yourself. If you are an expert in organic chemistry and you work here on organic solar cell

development, I think this is a fascinating study for you, especially if you're looking at further down the road at the relationship between the ring-level molecular property and the device-level performance; how you can optimize here the complex device buildup. But this would be the topic of another

video. Here also quite a lot of computer simulation. I don't know how far we can integrate AI design properties already here in the device development, but you see there's almost no area here in this topic where AI could not potentially have an impact on the performance.

of this. Just want to make sure that we understand it. If we would code it here, this skeleton code, this simplification by J-GPD4. What's happening here in the code is easy. If we build now this class of a RingFormer layer, what we do, we start

Here, with the atom-level GNN, we have a message-passing algorithm. This process uses the atomic-level graph to capture local atomic neighborhood features. Then, we build a ring-level cross-attention mechanism from our Transformer, using a transformer-based cross-attention mechanism to model the global

relationship between the different atoms and components of our rings. Then, we bring it together in an inter-level graph neural network for atom-ring interactions. Then we have a fusion layer. So you see this is exactly what we went through, so I wanted

to make sure that the explanation, the simplified explanation, and the code structure go hand in hand. And then, just to remind you, this cross-attention is crucial. Please take care about this, because it captures the global structural relationship between the rings in a molecule or compound. It

incorporates the edge attributes shared between atom and bond types in the attention computation. And we can model the long-range dependencies between the rings much more computationally efficiently, especially if we build on virtual nodes, but more about this later. So here you have, from

ChatGPT4 Omni, the complete code: atom-level GNN, message passing, ring-level cross-attention, inter-level hierarchical fusion, initialization, and now forward. Great! If you're not sure how cross-attention works, I've reminded you here. Great! But I already mentioned this: add a

virtual node for the global context. Now, this is a kind of trick if you want to capture global interaction with dense computation. We simply add a virtual node that connects to all ring nodes. This allows for a very elegant

Global information aggregation without the computational overhead of calculating a full attention mechanism. So, let's see a quick and dirty method that works. What are the benefits? We have an edge-aware attention component here. We have a global context via our virtual node, and we have

a flexible neighborhood attention mechanism focusing on immediate neighbors and localized attention, while also incorporating the global context via this virtual node. So we have the best of both worlds. And yeah, I think this is it. I just wanted to give you this interlude in my normal

videos, because I think it's so beautiful. If you know how to build your own AI system, you can do this with elegant ease. The coding is done, more or less, with beautiful code editors, whatever you prefer.

You just have to understand what you're looking for. In organic compounds, for example, you have to identify the critical element for your particular task—here, the ring structure for our organic solar cell molecules. Then you have to understand what kind of AI

neural network you want to apply. You could go with a graph neural network, a transformer, or a CNN. You now have to pick from this large toolset of possible models. You now have to understand how the model works,

how you can build interconnections between different model parts, and how you bring it all together. Honestly, as I've shown you, you can start with a small model and do the coding. You can run it on available public datasets first.

Approximation, and then you see immediately, because it is open source and it is your code that you're executing here, you see immediately if something goes wrong. You can analyze the mistake; you can optimize your code because it's a transparent system, it's not a black box system. And I think

this is the way forward here for dedicated expert systems in AI. And yeah, my main message is we should design our AI system for our specific task. I've given you the example here of organic solar cells by using this approach of modeling our AI, where we have

reusable components—components that we already tested, components that we know how to interpret, components that we know how to combine with other components. I think this makes AI so much more interesting, so much faster, more elegant, more powerful. And really, if it's open source, you can have it on your local

machine. You can have all your personal data integrated if you're working here in a laboratory, in a university, in a commercial company. You really should be aware that you keep your experimental data in-house; you don't send it off to some cloud. But maybe you have here a

closed parameter where you share your data, and then you can build your own models here locally with a model AI system. I hope you are a little bit excited; I hope you found some new ideas, maybe. That would have been great, and maybe I'll see you in my next video.

Monolithic AI vs Modular AI

A pure research channel (R&D). Code new Artificial Intelligence (AI) models with @code4AI - where complex AI concepts are demystified with clarity grounded in theoretical physics. Delve into the latest advancements in AI, from transformative Visual Language Models and Graph-Based AI Systems to cutting-edge Text-to-AI Game and interactive Video Environments. 

Our explanatory content bridges the gap between theoretical physics, computer science, and AI, offering deep insights into Large Language Models (LLMs) to Vision Language Action (AI Robotics). Discover new mathematical models powering the core of AI. New AI World Models. 



Discover AI

소개 특정 작업을 위한 모듈형 AI 시스템 소개

특정 애플리케이션을 위해 설계된 새로운 모듈형 AI 시스템을 소개합니다.

기존 독점 AI 시스템(예: OpenAI)과 대조하여 특정 요구사항에 대한 AI 이해 및 코딩의 이점을 강조합니다.

모듈식 오픈 소스 AI 시스템 구축에 중점을 둡니다.

신뢰할 수 있는 모듈형 AI 시스템의 핵심으로 사양을 강조하는 최근 간행물을 언급합니다.

특정 업무에 맞춘 소규모의 로컬 고성능 전문가 시스템을 옹호합니다.

응용 분야: 유기 태양전지 분자 특성 예측

분자 특성 예측을 위해 그래프 신경망(GNN)을 사용합니다.

고차 분자 구조를 처리하는 데 있어 기존 GNN의 한계를 해결합니다.

경쟁사 대비 22% 향상된 전용 AI 시스템을 설명합니다.

특정 작업을 위한 전용 타겟팅 AI 시스템의 장점을 강조합니다.

AI 시스템 설계에서 명확한 사양의 중요성을 강조합니다.

모놀리식 AI 시스템의 문제점과 제안된 솔루션

소수의 대기업이 통제하는 모놀리식 AI 시스템의 한계를 비판합니다.

복잡한 AI 시스템 구축의 진입 장벽을 낮추기 위한 모듈식 시스템 설계를 제안합니다.

모듈형 AI 시스템에서 재사용 가능한 구성 요소의 이점을 강조합니다.

모듈성을 보다 안정적인 AI 시스템 구축과 연결합니다.

유기 태양 전지(OSC) 분자를 최적화하기 위한 주요 특성을 지정하는 개념을 소개합니다.

OSC의 주요 속성(전력 변환 효율, 밴드 갭 에너지 등)을 나열합니다.

유기 반도체 분자에 대해 사용 가능한 데이터 세트에 대해 설명합니다.

AI 시스템을 구축하기 전에 목표와 작업을 정의할 필요성을 강조합니다.

그래프 신경망 및 관련 개념에 대한 학습 리소스를 제공합니다.

모델 아키텍처 및 구현 세부 사항

모델의 입력 그래프, 이웃 정의 및 마스킹에 대해 설명합니다.

특정 분자 속성을 예측하는 작업을 설명합니다.

향상된 결과를 위해 엣지 속성을 통합하는 확장 네트워크에 대해 언급합니다.

모델 코딩을 위한 리소스 링크를 제공합니다.

유기 태양 전지를 위한 모듈식 AI 시스템 설계

분자 화합물의 복잡한 방향족 고리 문제를 해결합니다.

다양한 고리 구조(방향족, 헤테로 방향족, 융합)를 설명합니다.

원자 수준 및 링 수준 모듈이 있는 모듈식 시스템의 설계에 대해 설명합니다.

링 레벨 교차 주의에 트랜스포머 모듈을 사용하는 방법에 대해 자세히 설명합니다.

레벨 간 GNN 모듈과 퓨전 레이어에 대해 설명합니다.

파이토치 지오메트릭과 GPT를 사용하여 코드를 생성하는 방법을 보여줍니다.

원자 수준의 GNN 프로세스 및 코드 구현에 대해 자세히 설명합니다.

사용 가능한 라이브러리를 사용하여 코드를 쉽게 구현할 수 있음을 강조합니다.

링 수준의 교차 주의 메커니즘에 대해 설명합니다.

이분 그래프 구조를 사용하여 레벨 간 메시지 전달을 설명합니다.

AI 시스템의 예측 목표를 설명합니다.

태양전지 효율에서 링 시스템의 중요성에 대해 설명합니다.

링 시스템이 전자 및 구조적 특성에 미치는 영향에 대해 설명합니다.

빛 흡수, 전하 수송 및 안정성 측면에서 방향족 고리의 이점을 강조합니다.

계층 그래프 시스템을 원자, 링 및 인터레벨 그래프로 요약합니다.

유사한 접근 방식의 공식 간행물을 참조합니다.

발행물에 표시된 대로 모델의 레이어를 자세히 설명합니다.

모델에 여러 레이어가 있을 수 있음을 언급합니다.

출판물 세부 정보를 제공합니다: 홍콩 폴리테크닉 대학교의 링포머.

코드 구조를 요약하면 원자 수준의 GNN, 링 수준의 교차 주의, 수준 간 융합입니다.

글로벌 관계를 포착하는 데 있어 교차 주의의 중요성을 강조합니다.

글로벌 컨텍스트를 위한 가상 노드 추가를 언급합니다.

글로벌 정보 집계에 가상 노드를 사용할 때의 이점을 강조합니다.

핵심 요소 식별, 신경망 선택, 모델 작동 방식 이해, 구성 요소 결합 등 맞춤형 AI 시스템을 구축하는 과정을 요약합니다.

Monolithic AI vs Modular AI

Monolithic AI vs Modular AI

스크립트