AI as we know it today is actually quite dumb. Yes, this includes chat GPT stable diffusion Sora and all the other state-of-the-art models that we have right now. They're still very incapable and inefficient, and the future generation of AI will look very different from what we have now. So in this video, I'm going to explain why the current generation is so limited and what the future generation of AI will look like. First, we need to understand the mean mechanics of AI as we know today. All AI is based on the neuron Network, which is designed based on the human brain. This is basically a network of nodes in which information flows through from one end to the other. Now, this is going to be a very simplified explanation of how a neural network works. I'm explaining this for people without a technical background in AI. So if you do have experience in AI, feel free to skip this section. Each do in a neural network is called a node or neuron, and each line of nodes is called a layer. You might have heard of the term

deep learning or deep neuron networks. This is basically a neuron network with many layers, hence it is very deep. Each node determines how much information flows through to the next layer. Now, again, this is an oversimplification. There are a lot of settings like weights and biases and activation functions, but basically just think of this neuron network as a series of dials and knobs which determine how much information flows through to the next layer. Here's a simple example. Let's say we have this neuron network which is designed to determine whether an image is a cat or a dog. For its input, we would feed it an image of a cat or a dog, and this image would be broken down into data, also known as tokens, which are then fed through this neural network. Eventually, after the data flows through all these layers, it reaches the end layer which would conclude whether the image is a cat or a dog. Now, what about training a model? How does that work? Well, a neural network needs to undergo usually millions of rounds of training to learn how to do something. Here's an example of

how one round of training would look like. Let's say you input an image of a dog, and then this image would be broken down into data which flows through this neuron Network, and it spits out the answer, this is a dog. Well, in that case, since it got the answer correct, it's likely that these dials and knobs, which we can also refer to as weights, are set correctly. If it gets the answer right, well, we don't really need to tweak these weights further. However, what if it gets it wrong? What if it says that this is a cat? Well, in that case, it would incur a penalty, and this penalty would cause the weights in this neuron Network to be updated so that this penalty would be minimized in the future. Specifically, the weights would be updated from the last layer to the next layer, back to the next layer back in a process which is called backpropagation all the way until it reaches the first layer of nodes. And usually, one round of training isn't good enough, so the network would undergo

millions of rounds of training, where the weights would be slightly tweaked to minimize the penalty incurred from any errors. And this goes on and on until finally we reach the configuration of dials and knobs, so that this neuron Network can very accurately determine whether any image is a cat or a dog. And this is how AI models that we know today are trained as well. So, for example, GPT is basically a neuron network, but these dials and knobs are optimized for understanding natural language. Stable diffusion is another neuro Network, where the dials and knobs are optimized for image generation. Now, again, this is very much an oversimplification, and the architecture or basically the design of the neuron network is also very important. For example, how many layers should we have, how many nodes in each layer should we have. There are also many different architectures, such as the Transformer model for large language models, or LSTM for time series data, or convolutional neuron networks for object detection and image classification. But in a nutshell, the backbone behind all these AI models is just a neural

network, which has a preconfigured set of dials and knobs to do the job accurately. So now that you understand how the current generation of AI works, let's look at the biggest limitations of this. First of all, once the model is finished training, the weights or basically these dials and knobs are fixed in value. When the user asks ChatGPT something or when the user uses Stable Fusion to generate an image, these dials and knobs do not change in value. In other words, all the AI models that we have today are fixed. Think of this as a brain that cannot learn or get any smarter. For example, GPT-4 cannot continue learning and become smarter and smarter with time. If we want a smarter model, well, we need to train a new generation of GPT, such as GPT-40 or GPT-5, or whatever you want to call it. Same with Stable Diffusion, for example, Stable Diffusion 2 cannot get smarter and generate better images as we use it more and more. In order for it to improve, we currently need to train a new

generation also known as stable diffusion 3. And once stable diffusion 3 is finished training, well, that's as smart as it gets. And if you don't think it's good enough, well, you need to train a new model. So basically, all the AI models that we have today are fixed in their intelligence and their capabilities. Again, think of this as a brain that has stopped growing and cannot learn or get smarter. But this is not how the human brain works. There's a term called neuroplasticity, which refers to how the brain can reorganize or reconfigure itself by forming new neural connections over time in order to adapt to new environments or learn new things. And that's exactly what the next generation of AI can do, which we'll talk about in a second. But there's another huge limitation of current AI models. They are extremely inefficient and compute-intensive. As you may know, AI is designed based on the architecture of the human brain. So let's compare it to the efficiency of the human brain right now.

GPT-3 has 175 billion parameters. This was trained using thousands of GPUs over several weeks or several months. The total power required for training GPT-3 was estimated to be around 1,287 megawatt hours of electricity. This is roughly equivalent to the monthly electricity consumption of 1,500 homes in the USA. Now, keep in mind, GPT-3 was completed in 2020, that's 4 years ago. The latest version, GPT-4, is closed source, so we don't actually know its architecture or how long it took to train, but we do know that it has around 1.76 trillion parameters, 10 times more than GPT-3. Keep in mind that the amount of computations required scales exponentially as the parameter size gets larger, so from a rough calculation, GPT-4 could have taken around 41,000 megawatt hours of energy to train. That's enough energy to power around 47,000 homes in the US for a month. The compute used to create.

these state-of-the-art models that we know today, such as GPT-4 or CLAE-3 or Gemini 1.5 Pro, require massive data centers and a lot of energy. That's why Tech Giants are scrambling to invest and build even bigger data centers because they know that compute is the main limitation here. That's exactly why Microsoft and OpenAI are planning a $100 billion Stargate project to build the biggest data center in the world. All of this is for more compute. Now, contrast this to the human brain. Some might say the human brain is still more intelligent than GPT-4, at least in some regards. The human brain only uses 175 kilowatt hours in an entire year, and it gets this energy in the form of calories from the food we eat. So, training GPT-4 is estimated to require approximately 234,000 times more energy than what the human brain uses in an entire year. In other words, the energy required to train GPT-4 once could power the human brain.

for over 234,000 years now, I gave this comparison to show you that there's something fundamentally wrong with AI models today. They are very energy inefficient and they take up a lot of compute. It's not even close to the efficiency of the human brain. So the next generation of AI has to solve this efficiency problem as well, otherwise it will not be sustainable. So to summarize, the major limitations of current AI models are: number one, they are fixed and unable to improve or learn further after being trained, and number two, they're also very energy intensive and inefficient. These are the two biggest problems of the current generation of AI. Now let's enter the Next Generation. We aren't there yet, but there are a few possible architectures that are being discussed and developed as we speak. The first architecture is called liquid neural networks. Now, liquid neural networks are designed to mimic the flexibility or the plasticity of the human brain. The human brain is very flexible and can reorganize or reconfigure itself over time, and this ability allows the brain

to adapt to new situations or learn new skills or compensate for injury and disease. For example, when you learn something new, your brain changes structurally and functionally to accommodate the new information. Learning a new language can lead to changes in the brain structure and function, such as increased density of gray matter in the left hemisphere. The brain can also reconfigure itself to recover from injury. For example, after a traumatic brain injury, physical therapy and cognitive exercises can help rewire the brain to regain lost functions. And for people who've lost a sense like sight or hearing, the brain will reorganize itself to compensate for the loss and make other senses become more acute. So, this flexibility, this plasticity, is exactly what liquid neuron networks are designed to have. Liquid neuron networks can adapt in real time to new data. This means that the configuration of the neuron network can change as it receives new inputs. And that's why it's called liquid. These connections in the network and these dials and knobs are fluid so they can change dynamically over time. Liquid

neuron networks also retain what they have learned while incorporating new information. This is similar to how our brains can remember old information while learning new things. So here's how liquid neuro networks work. They have three main components. Much like a traditional neuron Network, it has an input layer which receives the input data. But then in the middle, we have this liquid layer, otherwise known as a res res. This is the core component of a liquid neuron Network, and it's basically a large recurrent neuron Network. Think of this as a big bowl of water in which each Splash creates a ripple. These ripples are basically the neurons in this network reacting to inputs. The reservoir acts as a dynamic system that transforms the input data into a high-dimensional representation called Reservoir States. The reservoir's rich dynamics and transformations capture the complex temporal patterns in the input data. And then finally, we have the output layer. This layer receives the reservoir States and Maps them to the desired output using what is called a readout function. In layman terms, this is a layer that looks at the ripples in the

reservoir and tries to understand what it all means. It takes the dynamic patterns from the reservoir and makes predictions or decisions from it. The key aspect of liquid neural networks is this Reservoir layer, which remains untrained during the entire learning process. Only the output layer is trained to map the reservoir states to the target outputs. In other words, to understand what these ripples mean. And because this Reservoir remains fluid and flexible throughout time, it's not fixed in value. That allows this liquid neural network to basically adapt to new data and learn new things. Here's how you would train a liquid neural network. The connections between neurons in their reservoirs are set up randomly at the start. These connections typically stay the same and don't change during training. Next, you would feed the input layer some data. And when this data is broken down into tokens and it reaches the reservoir layer, it causes the neurons in the reservoir to react and create complex patterns, much like ripples in water. So as this input data creates ripples, you basically observe

and analyze the patterns created in the reservoir over time. And that's exactly what the readout layer does. It learns to recognize these patterns. It's like learning, Ahuh, this is what caused this type of ripple, and that is what caused this other type of ripple. And eventually, after lots and lots of rounds of training, the readout layer can make accurate predictions based on observed patterns. Again, note that only the readout layer is trained, which is simpler and faster because you're not adjusting anything in the reservoir layer. This is much quicker and needs less compute compared to traditional neuron networks. That's because in neuron networks that we know today, all the weights, including those in the hidden layers, are trainable. This means more parameters to optimize, leading to longer training times and higher computational requirements. But in liquid neuron networks, you don't adjust the weights of the reservoir during training. Only the readout layer is trained, and this significantly reduces the computational burden during training since fewer parameters need to be optimized. Plus, it's a lot faster to train. Thanks to our

sponsor bright data. Bright Data is an all-in-one platform designed to help businesses collect high-quality web data at scale. This is especially useful for AI companies, which require huge amounts of diverse and high-quality training data to build robust and unbiased AI models. Collecting this training data manually can be time-consuming and prone to errors, and that's where Bright Data comes in. With Bright Data, you can access high-volume, high-quality web data effortlessly - from parsed, validated data sets to custom scraping solutions, they've got you covered. Get parsed and clean data sets ready to use on demand, customize any data set to fit your specific needs, and benefit from reliable delivery and full compliance. In fact, every 15 minutes, their customers scrape enough data to train ChatGPT from scratch - that's a lot of data to say the least. They have many tools like the web scraper API, the proxy manager, and unblocking technologies to help automate your data scraping at scale, allowing you to build reliable data sets to train any AI or LLM. Visit the link in the

description below to learn more. It's a lot faster for these liquid neuron networks to converge at an optimum. And because of this reservoir where the weights and configurations can change dynamically depending on the data that you feed it, liquid neuron networks can potentially be much smaller than traditional neuron networks, which have fixed weights and connections. This offers a lot more efficient learning and inference. So, for example, researchers at MIT were able to pilot a drone using a liquid neuron network with only 20,000 parameters, which is very tiny compared to state-of-the-art AI models such as GPT-4, which often have over a trillion parameters. Just think about that, 20,000 parameters versus over a trillion parameters. So, these smaller sizes generally translate to faster inference and lower computational requirements. Liquid neuron networks are also way less memory intensive. Again, since you don't train the reservoir weights, memory usage is much lower during training compared to traditional neuron networks, where the gradients and the parameters for all layers must be stored in memory. Liquid neuron networks are particularly good at

processing temporal data due to their Dynamic Reservoir, so they excel in tasks that involve complex time series data. Now you might be wondering, well, how can these liquid neuron networks actually be applied in the real world? So here are some use cases. As we race to build fully autonomous AI robots, these robots will be deployed in the real world, and often times they might encounter situations that they've never seen before during training. For example, there could be unpredictable environments in search and rescue missions. But with liquid neuron networks, these robots can adapt to changing conditions and learn new tasks on the fly. And eventually, we're going to have these autonomous robots in our houses, helping us do chores and other tasks. But maybe you have a certain way of folding clothes or doing the laundry or cooking that the robot was never trained on. So with a traditional neuron network, these robots aren't able to learn new skills after being deployed. But with liquid neuron networks built into a humanoid robot, it can learn new tasks that you teach it, and this robot will become a lot more personalized for

you and then we have autonomous driving. There's no doubt that self-driving cars will eventually become the future, but current Technologies still do not perform well, especially in challenging environments or new conditions. Again, this is because traditional neuron networks can only do well on data that they were trained on. They're not able to adapt to new environments. But with liquid neuron networks, autonomous vehicles can navigate complex and dynamic environments by continuously learning and training from sensor data and adjusting their behavior accordingly. It's constantly training and improving over time. Now, as I've mentioned before, liquid neuron networks often incorporate recurrent connections, making them suitable for processing time series data, so it's great for things like weather prediction and of course stock trading. The stock market is filled with Ever-Changing Trends and Cycles, so it's close to impossible for one fixed algorithm or formula to beat the market. However, because liquid neural networks can adapt to everchanging data, it can optimize trading strategies in real time to maximize profits. In other words, you could be constantly streaming the latest Market data to this liquid neuron.

Network, which would change its configuration to adapt to this data in real time, to help you maximize profits. Another use case would be Healthcare: liquid neuron networks can be used in wearable devices to monitor patients in real time, adapting to changes in the patient's conditions and predicting potential health issues before they become critical. In cyber security, liquid neuron networks can continuously learn from Network traffic and user Behavior to adapt Access Control policies and detect anomalies or unauthorized access access attempts. Yet another use case would be streaming services such as Netflix: they can use Liquid neuron networks to adapt to each user's viewing habits and preferences, providing more personalized content recommendations. Another use case would be smart City management. For example, liquid neuron networks can optimize traffic flow in real time by learning from traffic patterns and changing traffic lights accordingly to reduce congestion and improve efficiency. Energy management is also very relevant: smart grids can use Liquid neuron networks to Balance power supply and demand in real time, improving efficiency and reducing costs by adapting to consumption patterns. However, although liquid neuron networks seem

promising. It does have its limitations. This is still a relatively new concept in the field of neuron networks, and research on them is still in its early stages compared to more traditional architectures. While liquid neuron networks show promising theoretical benefits, such as the ability to process continuous data streams and adapt on the fly, there is still a lack of real-world results demonstrating their superiority on a large scale. Many researchers are likely waiting for more compelling benchmark results before investing significant effort into liquid neuron networks. Also, as I mentioned previously, they're particularly suited for temporal or sequence data. So, for tasks that do not involve time, such as identifying images of cats or dogs, traditional neuron networks might actually be more effective and straightforward to implement. Also, the dynamics within this reservoir layer can be very complex and difficult to interpret, and this makes it challenging to understand how the reservoir processes these inputs. It would be quite hard to fine-tune it for optimal performance. Finally, there is a lack of standardized support and fewer established frameworks for liquid

neuron networks compared to traditional neural networks, and this can make implementation and experimentation more challenging. So, all in all, liquid neuron networks are still a very early concept and an area of active research. Unlike traditional neuron networks that are fixed and need to be retrained with a large data set to learn new information, liquid neuron networks can update their knowledge incrementally with each new piece of data. This offers a flexible and adaptive model, which could potentially become infinitely smarter over time. Now, liquid neuron networks aren't the only possibility that could become the next generation of AI. We have another type of neuron network, which is designed to mimic the human brain even more than traditional neural networks, and this brings us to spiking neuron networks. These are closely inspired by the way neurons in our brains communicate using discrete spikes or action potentials. You see, in the human brain, which is basically a network of neurons, each neuron doesn't immediately fire to the next set of neurons when it receives input. Instead, the input has to build up.

to a certain threshold. And once it passes this threshold, then it fires to the next set of neurons. And after it fires, it goes back to its resting state. Well, spiking neuron networks are designed to mimic this behavior. So here's how it works. The architecture is quite similar to traditional neuron networks. However, for each neuron, it waits to receive signals or spikes from other neurons. Think of these spikes as like little electric pulses. The input data, such as an image or a sound, is turned into these spikes that move through this neural network. For example, if it's a loud sound, it might generate more spikes, while a quiet sound might generate fewer spikes. Now, each neuron in the network collects incoming spikes. Imagine a bucket collecting drops of water. As more spikes come in, the bucket fills up. And when the neuron gets enough spikes, in other words, when it reaches a certain threshold, it fires a spike to the next set of neurons. And after firing, it resets and starts collecting again from zero. So instead of using continuous signals like traditional neuron networks, spiking neuron networks use spikes.

which are basically bursts of activity at discrete time points. To process information, in other words, spiking neuron networks incorporate time into their processing, with neurons firing only when their potential exceeds a certain threshold. Now, there are different methods and algorithms to train a spiking neural network, and there currently isn't a standard way that's set in stone, so this is still an active field of research. One common method is called Spike timing-dependent plasticity or STDP. This method is inspired by how the brain strengthens or weakens connections between neurons. So, if one neuron spikes just before another, then the connection between them gets stronger. If it spikes just after, then the connection gets weaker. It's like learning which connections are important based on the timing of the spikes. And speaking of timing, it's the exact timing of spikes that matters. It's not just about how many spikes there are, but when they happen. Now, STDP is only one method to train the spiking neuron networks. There are a few other ones which are beyond the scope of this video.

but like traditional neuron networks, spiking neuron networks have to undergo millions of rounds of training with a lot of data, and eventually, the configuration of the network and all its parameters will reach an Optimum State. Now, again, I'd like to remind you that this is a very simplified explanation of spiking neuron networks, and I've left out a lot of mathematical details, but in a nutshell, that's how it works. So, you might be wondering, well, what are the benefits of spiking neural networks? First of all, it's designed to mimic the human brain even more by implementing this spiking mechanism. So, in theory, maybe we could reach a superior level of intelligence compared to the current generation of AI if we mimic the human brain even more. But the biggest benefit of spiking neuron networks is their efficiency. If you remember at the beginning of the video, I compared the energy consumption of the human brain versus a current state-of-the-art model like GPT-4, which requires huge data centers and huge amounts of compute. That's because traditional neuron

networks are always active. Each input of data activates the entire neural network, so you have to do an insane amount of Matrix multiplications across the entire network just to do one round of training or inference. However, for spiking neural networks, they only use energy where spikes occur, while the rest of the neuron Network remains inactive. This makes it a lot more energy efficient. Plus, spiking neuron networks are particularly suitable for neuromorphic chips, which are designed to mimic the human brain. Now, neuromorphic chips are a huge topic and deserve its own full video, so let me know in the comments if you'd like me to make a video on this as well. So how can these spiking neuron Networks actually be applied to the real world? Well, because these neuron networks can encode and process information in the timing of spikes, this is great for processing temporal data. This makes them great for adaptive and autonomous systems. Plus, this Spike timing-dependent plasticity, which I mentioned before, where the timing of the spikes influences the strength of the

connections in the network. This can lead to more robust and adaptive learning capability. So, this dynamic learning can make spiking neuron networks suitable for autonomous systems, such as self-driving, where the AI has to learn and adapt to changing environments. Or it can be used in real-time processing, like predicting the stock market, or patient monitoring, and personalized medicine. And of course, autonomous robots. Now, although spiking neuron networks offer some huge benefits, especially regarding energy efficiency, they do have some limitations. Setting up and programming spiking neuron networks is more complicated compared to traditional neuron networks. This spiking behavior, of course, adds a layer of complexity, making them harder to design and understand. Training spiking neuron networks is also quite difficult. Current neuron networks use methods like backpropagation to adjust their parameters, but this process doesn't work well with these discrete time-based spikes. Researchers are still trying to find an effective training algorithm for spiking neuron networks. Also, given this additional dimension of time, spiking neuron networks might actually require more computational resources to simulate. This is because

they need to track and process spikes over time, which can be computationally expensive. Yet another limitation is that running spiking neuron networks efficiently often requires specialized hardware, such as neuromorphic chips, which are not widely available or standardized compared to conventional computing hardware. Neuromorphic chips are optimized for this spike-based processing and are still being developed. That's why, for example, Sam Alman is investing millions of dollars into a neuromorphic chip company called Rain. Finally, while spiking neuron networks show promising results, especially for time-based data, they often lag behind current neuron networks for non-time-based data. They often underperform compared with current AI models, particularly for complex tasks. This is partly due to the challenges in training spiking neuron networks effectively. And as with liquid neuron networks, spiking neuron networks are also relatively new, so there are fewer tools and frameworks available for developing spiking neuron networks compared to current AI models. This makes experimentation and development slower and more difficult. But anyways, that sums up what could

potentially be the next generation of AI. To bring it all back, the current generation of AI is very energy inefficient, requiring huge amounts of compute. Plus, it can't learn new things after being trained. If we want to achieve AGI or ASI, we need to essentially create something as efficient and as fluid as the human brain, which can constantly learn new things and adapt to changing environments. These are the two essential things that new types of neuron networks, such as liquid neuron networks and spiking neuron networks, can solve, at least in theory. However, these are still relatively new, and they are still being developed. But the potential could be massive. Imagine an AI that can keep learning and get infinitely smarter. Let me know what you think about these neuron networks in the comments below. Things are happening so fast in the world of AI. It's quite hard to keep up with all the technological innovations that are happening right now. So, if I've missed any other groundbreaking architectures that are worth mentioning, please let me know in the comments below.

and I'll try to do a video on that. As well as always, if you enjoyed this video, remember to like, share, subscribe, and stay tuned for more content. Also, we built a site where you can find all the AI tools out there, as well as find jobs in machine learning, data science, and more. Check it out at ai-search. Thank thanks for watching, and I'll see you in the next one.

The future of AI looks like THIS (& it can learn infinitely)

Making AI easy to understand for everyone. Subscribe to stay on top of the latest AI news, trends, and tools!

For all business inquiries please email me ↓


The future of AI looks like THIS (& it can learn infinitely)

The future of AI looks like THIS (& it can learn infinitely)

스크립트