Understanding Hopfield Networks: Associative Memory and Learning

Hopfield networks efficiently store and retrieve memories by minimizing energy through local configurations, mimicking biological processes.

### Understanding Memory Retrieval through Hopfield Networks
* 🎶 At a party, hearing a snippet of your favorite song can trigger **instant recall of lyrics and related memories**, showcasing the brain's **impressive associative memory capabilities**.
* 🧠 The challenge lies in how the brain **efficiently retrieves this information** without performing exhaustive searches through an **astronomical database of sounds and memories**.
* 🔬 Insights from molecular biology, particularly Levinthal's paradox of protein folding, highlight how proteins **quickly find their optimal configurations** by **minimizing potential energy**, a process that can inform our understanding of **memory retrieval**.
* ⚡ Similarly, Hopfield networks, developed by John Hopfield in 1982, offer a model for associative memory by structuring information storage in a way that allows for **efficient recall through energy minimization**.
* 🕹 In these networks, neurons represent states that can either **align or misalign** based on connection weights, and the system evolves towards configurations that **minimize energy**, effectively **recalling memories**.
* 📉 The learning process involves setting connection weights according to the **Hebbian Learning Rule**, enabling the network to **store multiple patterns**, though the capacity is limited to **approximately 0.14 times the number of neurons**.
* 🔍 Despite their limitations, Hopfield networks serve as a foundational model for understanding **associative memory**, paving the way for more advanced systems like **Boltzmann machines**.
* 📚 The video concludes with a recommendation for Shortform, a platform that **enhances reading comprehension and learning** through comprehensive book guides and insights.

Consider the following scenario. You are at a party when you hear a short snippet of your favorite song. (song plays) Almost instantly, your brain recalls the lyrics of that song and many related memories, such as attending a recent concert featuring that artist. It seems very natural and unimpressive. After all, people can recall information all the time. However, if you think about it, this problem is computationally non-trivial. Let's put ourselves in the shoes of evolution and try to come up with an algorithm for the brain to solve it. The first approach that comes to mind is to actually store some kind of a database of all the songs you have heard a sufficient number of times, along with related information like the title and lyrics. When an audio fragment of an unknown song is received, we can scan through all the songs in our database, find the one that has a close enough match, and retrieve its lyrics. However, the search space of every sound I've ever heard is astronomically large.

And it's even larger when considering every single memory you have formed since childhood. Performing an exhaustive search would be simply impossible. Yet, you seem to have no problem instantly recognizing familiar stimuli and finding associations between them. So, how does the brain accomplish this so quickly? In this video, we will lay the foundation for a new paradigm of information storage and retrieval, which is more in line with biology, and actually build one of the simpler models of this process known as Hopfield networks, developed by John Hopfield in 1982, who laid an important groundwork for many ideas in both neuroscience and machine learning. If you are interested, stay tuned. Just to reiterate, we need a way to somehow query what we know and find associations between existing memories and new inputs without explicitly checking individual entries for a match, which seems like an impossible problem. However, we can draw insights from a seemingly unrelated field of molecular biology and, in particular, a concept known as Levinthal's paradox of protein folding.

As you might know, proteins are long chains of amino acids that fold into specific 3-dimensional structures, which determine their function. The number of possible structural configurations a protein can take, considering all the different ways you can arrange the atoms of an amino-acid chain in 3-dimensional space, is absolutely enormous. Given the number of possibilities, it seems like it would take an astronomical amount of time for a protein to surge through all the possible structures to find its correct folded state. In fact, there are computations showing that even if the protein samples different conformations at a nanosecond scale, it would still require more time than the age of the universe to arrive at the correct configuration. Yet, in reality, proteins fold into their native structures in a matter of milliseconds. So how do they accomplish this? When I first heard this paradox in high school, it seemed to me like an ill-posed question. After all, the protein molecule is not a computer. So it doesn't do any sort of search.

It just folds into the most stable and favorable configuration according to physical laws. This is similar to how when you throw a ball, the ball doesn't search through all the possible trajectories to select the optimal parabolic one. It simply follows that path because...well, physics works this way. But how can we think about this folding into a favorable configuration? Favorable for what, exactly? Let's introduce the concept of energy, as it will come in handy in future videos as well. If you think back to your high school's physics days, you may recall something along the lines of energy is a quantitative property that describes the state of a system, namely the capacity to do work or cause change. Energy can be stored in a variety of different forms, and for the case of proteins, we will be interested in potential energy stored in the interactions between the atoms in the protein chain. Each possible configuration of the protein chain has a specific potential energy level determined by the sum of all of these atomic interactions.

In other words, we can assign a positive number to each state equal to its energy, which is a function in some very high-dimensional space where different dimensions correspond to degrees of freedom you need to uniquely describe a configuration. For example, all possible dihedral angles of peptide bonds. Let's abstractly visualize it as having just 2-dimensions. Then the energy function can be thought of as a surface where each point on it represents a possible protein configuration, and the height of the point represents the potential energy of that configuration. This is what we are going to refer to as Energy landscape. For a protein, it could be a complex rugged surface with many peaks and valleys. Now, here's the key point. A protein molecule, like any physical system, tends to minimize its potential energy guided by the second law of thermodynamics. It will naturally seek out the configuration that has the lowest possible energy level, as this represents the optimal arrangement of its atoms.

and this, in fact, corresponds to the native correctly folded state. When a protein is folding, it is essentially rolling downhill on the energy landscape, following the steepest path towards the valley. This is why proteins can fold so quickly. They don't need to search through all possible configurations. They simply follow the natural tendency of physical systems to minimize their potential energy. The protein's folding process is guided by the shape of the energy landscape, which, in turn, is determined by the interaction between its atoms, and the descent along the surface is essentially driven by the underlying physical process of energy minimization. Now, the core idea is to achieve something similar for the case of associative memory. Suppose we have a system that can encode information in its states, and each configuration has a specific potential energy determined by the interaction between the states. Then we need to, first, somehow sculpt the underlying energy landscape so that memories or state patterns we want to store correspond to local minima, these wells in the energy surface.

Second, we need something that would play the role of the second law of thermodynamics and would drive the changes in the states directing the system towards the nearest local minimum. Once these two things are achieved, retrieving a memory that is most similar to the input pattern is done by configuring the system to encode the input pattern initially and let then it run to the equilibrium descending into the energy well from which we can redoubt the source memory. Sounds neat, right? So let's get into building it. Let's consider a set of neurons which we can think of as abstract units that can be in one of two possible states, plus one or minus one. This is a simplified analogy of how nerve cells in the brain encode information through patterns of firing. They either generate an electrical impulse at a given point in time or remain silent. We'll focus on the fully connected network where each neuron has connections to every other neuron. These connections have weights associated with them. Real numbers that signify the strength of coupling.

between the corresponding pair of neurons. For a pair of units i and j, we denote the connection weight between them as w_ij and the states of neurons themselves as x_i and x_j. In the brains, connections between neurons or synapses have a well-defined direction. A pair of neurons is connected asymmetrically, meaning that the synapse from neuron A to neuron B is physiologically separate from the synapse that connects B to A if that one exists at all, and so they can have different weights. While we could generalize a Hopfield network to account for asymmetric connections, it would introduce complications and potentially unstable behavior. For simplicity, here we will stick to the original formulation of the Hopfield network, which assumes symmetric weights. In other words, neurons i and j are connected by the same weight in both directions. Now that we have a set of neurons symmetrically linked with each other through weighted connections, let's explore what these weighted connections represent. If w_ij is greater than zero,

The connection is said to be excitatory and favors the alignment between the two neurons. We can think of each connection as being happy or unhappy depending on the states of its neurons. For example, if w-ij is a large positive number, it means that neurons i and j are closely coupled, and one excites the other. In this case, when one neuron is active, the other tends to be active as well, and when one is silent, the other one is more likely to be silent. These configurations, where both x-i and x-j are either one or minus one, agree with the connection weight. However, if we observe, for example, that x-i is equal to one and x-j is equal to minus one, it conflicts with the excitatory nature of the connection, making such a configuration less likely. Conversely, when w-ij is negative, the connection promotes misalignment between the weights. This alignment between the signs can be expressed more concisely using the product x-i times x-j.

This product will be positive when both neurons have the same sign and negative when they have different signs. By multiplying this product further by the connection weight, we obtain an expression for the happiness of that connection. For a positive w-ij, happiness will be positive when the product of the two states is positive. But this is just one edge. We can extend this idea and compute the happiness of the entire network as a whole by summing this quantity across all edges. The larger that number is, the more overall agreement there is between connection weights and pairwise states of neurons. Ultimately, we will search for a set of weights that maximize this quantity, and maximizing happiness is equivalent to minimizing it with a minus sign, which you can think of as the measure of overall conflict between the actual configuration of states and what's favored by the connection weights. This total conflict between the weights and the pairwise states is exactly what we are going to define to be the energy of the system. As we discussed previously, we want the Hopfield network

to be able to gradually evolve towards energy minima. But looking closely at the formula, we can see that the energy value depends both on the states and the weights. So, there are a lot of things the system can tweak to change it. What exactly is getting adjusted? As we will see further, there are essentially two modes of network updates that nicely map to the two aspects of associative memory, namely adjusting the weights corresponds to shaping the energy landscape defining which configurations are stable by digging energy wells around them. This is the act of learning when we are writing new memories into the new network. Once the weights are fixed, tweaking the states of neurons to bring them into greater agreement with the weights corresponds to descending along the energy surface. This is the act of inference when we are recalling the memory that is at the bottom of the energy well which is nearest to the configuration of the input pattern. Let's take a look at inference first.

Suppose for a second, someone has already set the weights w and hands us the backbone of the network (the neurons themselves with all the connection weights). However, the exact configuration of states (which neurons are active and which are silent) is unknown. The question then becomes, How do we find the state pattern that would minimize the total energy?. As we discussed, simply checking all possible states is not an option. So, we will start with some initial state which could be either a partial or a noisy version of one of the memories or a random configuration altogether. Once the initial condition is set, we will iteratively try to lower the energy value by focusing on updating one neuron at a time. Let's denote the neuron we're currently considering as neuron i. We will calculate the total weighted input to it from all other neurons in the network. This input, which we'll denote as h-i, is the sum of the states of all other neurons multiplied by their respective connection weights. If h-i is positive,

It means that the weighted sum of the other neurons' states is in favor of neuron i being in the plus one state. Conversely, if h-i is negative, it suggests that neuron i should be in the minus one state to minimize the conflict with the other neurons. So, we will update the state of neuron i based on the sign of h-i. Notice that this update is guaranteed to decrease the energy of the network because from the 2 candidate states we are selecting the more energetically favorable one. You can think of this as a kind of voting process. Each neuron looks at the states of all other neurons weighted by the strength of their connections and decides whether to be active or silent based on the majority vote. We'll go through this process for each neuron in the network one by one, chosen in random order, updating their states based on the input from all other neurons. Once we've updated all neurons, we will have completed one iteration of the network inference and decreased the system's energy by a little bit.

We'll keep repeating this process, doing these sweeps through all neurons, updating them one at a time based on the current configuration. As we do this, the network will gradually evolve towards a configuration that minimizes the overall energy. At some point, however, we will reach a configuration where flipping any neuron would lead to an increase in energy. So no further adjustments would be necessary. At that point, the network has converged to a stable configuration where each neuron state agrees with the majority vote. This stable configuration represents a local minimum in the energy landscape. Now you might be wondering, is the network guaranteed to reach such a stable configuration? Could we possibly stumble into a particularly unlucky set of states and get stuck in a never-ending loop of flipping neurons back and forth? In other words, is such iterative flipping of one neuron at a time equivalent to doing a descent along the energy surface? This is where we come back to the point about symmetric weights. It turns out that there is a mathematical proof that I am not going to cover here,

stating that as long as your weights are symmetric, this simple majority vote single neuron update rule is guaranteed to eventually converge to a stable configuration if you do it enough times. To restate it, the Hopfield network can settle into different local minima based on its initial conditions. These local minima in the energy landscape correspond to distinct memories stored in the network. When we initialize the network with the pattern that is similar to one of these memories in some way and let it evolve, it will fall into the nearest local minimum, effectively recalling the complete stored memory, thus performing pattern completion or noise correction. But so far, we haven't talked about how we come up with this set of connection weights that encode specific memories in the first place. So, let's explore the learning process. Before we move to storing several memories, let's consider memorizing a single pattern of states. That means the network would have a single global minimum, one energy well, and would converge to the same pattern every time, no matter where you initialize it.

While it has little practical use, it provides a nice starting point to describe the learning procedure. Let's denote the template pattern that we'd like to store as csi, which is a vector packing the states of all neurons, and csi-i will denote the 'i'th component, the csi state of 'i'th neuron encoding the memory, while x-i refers to the state of 'i'th neuron in the network in general which could be tweaked. Revisiting our definition of energy, we want to set w-ij so that this quantity would be at its minimal value for the memory pattern. If we plug x-i equal to csi-i, we get the equation for the energy of the reference pattern as a function of weights which we want to turn into a global minimum. Notice that we don't really care about the absolute value of that energy as long as the energy of the desired memory pattern is less than the energy of any other configuration. Now, intuitively, the lowest possible energy is obtained

when all the connection weights fully align with the state pairs. But when we have just a single pattern, this is very easy to do. All we need is to set the weight, w-ij, to be the product of the corresponding pair of states in the memory pattern. This way, every connection is satisfied and the energy of the network when it's in the state csi becomes the negative of the total number of edges. When the network is in the state csi, any single flip of a neuron would increase the energy thus making it a stable state. I want to reiterate a crucial point here. If we want to come up with a set of weights that would dig an energy well around some pattern, then all we need to know are the pairwise relationships between states in that pattern. If the two neurons are active together in the source memory, strengthening the connection between them lowers the Hopfield energy of that memory effectively storing it in the weights for associative recall.

You might have heard the famous statement from neuroscience attributed to Donald Hebb: Neurons that fire together, wire together. And, in fact, what we just did is known as the Hebbian Learning Rule. Great! So we found a way to make a single pattern a stable state of the network. But we want to store multiple patterns. How do we do that? Here's the key idea. We can simply sum the weights we would get for each pattern separately. So if we have 3 patterns, csi-1, csi-2, csi-3, we can set the weights according to the following equation. What this will do is turn each of the patterns into a local minimum. It's pretty straightforward to show mathematically, and if you're interested, I encourage you to check out the references in the video description. However, intuitively, if the patterns you want to store are very different, so they are far away in the state space from each other

then if you first independently dig energy wells around each of them and then simply add the energy landscapes together, the resultant surface will have local minima in the same 3 valleys. And this nicely brings us to the limitation of the Hopfield networks. There is a limited number of valleys we can sculpt in the energy landscape before they start to interfere with each other. At some point, if we try to store too many patterns, the network will fail to converge to a stored pattern reliably and recall weird in-between kind of memories. The total maximum number of patterns you can store is thus limited and depends only on the size of the network. It is approximately zero point fourteen times the number of neurons. So if you have a Hopfield network of one hundred neurons, you can reliably store less than fourteen patterns in the best-case scenario. If you are unlucky, however, and some patterns are similar to each other or correlated, their energy will begin to interfere even before you reach the full capacity.

All of this makes vanilla Hopfield networks not useful for practical purposes. However, to this day, they provide a powerful and intuitive model of associative memory. A simple network of neuron-like units that can store and retrieve patterns through purely local learning and inference rules. Despite their limitations, Hopfield networks have laid the groundwork for more advanced energy-based models. In one of the next videos, we will look at the extension of the Hopfield networks known as Boltzmann machines. These generative architectures introduce additional hidden units and stochastic dynamics, allowing them to learn more complex probability distributions. There is also an extension to modern Hopfield networks, published in 2016 with John Hopfield himself as one of the authors, but that's a topic for another time. In the meanwhile, I'd like to take a moment and thank Shortform, who are kindly sponsoring today's video. Shortform is a platform that lets you supercharge your reading and gain valuable insights from books. Their unique approach of book guides

goes way beyond simple summaries by providing a comprehensive overview of the material. Not only do you get a concise version of the main points, but you also benefit from related ideas sourced from other books and research papers on the topic. They have an actively growing library of books from all sorts of genres such as Science, Health, and Technology. Not only that, but there is a useful AI-powered browser extension that allows you to generate similar guides for arbitrary content on the internet. Personally, I found Shortform to be really helpful, both when I am choosing books to read and writing notes and flashcards on the topic. Don't hesitate to bring your reading to the next level by clicking the link down in the description to get 5 days of unlimited access and twenty percent off on an annual subscription. If you like the video, share it with your friends. Press the like button and subscribe to the channel if you haven't already. Stay tuned for more computational neuroscience & machine learning topics coming up.

Goodbye and thank you for your interest in the brain. (Subtitles by Crimson Ghoul)  
Goodbye and thank you for your interest in the brain. (Subtitles by Crimson Ghoul)

A Brain-Inspired Algorithm For Memory

Artem Kirsanov

The Brain's Fast Information Retrieval and Protein Folding

The brain's ability to quickly recall memories and recognize familiar stimuli is computationally complex, making exhaustive searches impossible.

Hopfield networks, developed by John Hopfield in 1982, offer a biological-inspired model for information storage and retrieval.

The paradox of protein folding, with its enormous number of possible configurations, illustrates how systems can quickly find stable states without explicit searching, using potential energy from atomic interactions.

The energy landscape of a protein's folding process is visualized as a surface with peaks and valleys, representing different configurations and their potential energy.

Proteins minimize their potential energy by following the steepest path on this landscape, leading to their native folded state.

The concept of sculpting an energy landscape can be applied to associative memory systems, where local minima correspond to stored memories, and the system's dynamics drive it towards these minima for memory retrieval.

Neuron connections can be excitatory or inhibitory, influencing their alignment and activity.

Positive connection weights promote the same state in coupled neurons, while negative weights encourage misalignment.

The overall happiness of the network is calculated by summing the contributions from all connections, with the goal of maximizing this happiness and minimizing the conflict between states and weights, ultimately defining the system's energy.

Updating Neuron States in a Hopfield Network

The state of neuron i is updated based on the sign of its weighted sum from other neurons, with the goal of minimizing conflict and decreasing the network's energy.

This process is akin to a voting system, where each neuron updates its state based on the majority influence of others.

The network's iterative updates aim to converge to a stable configuration, representing a local minimum in the energy landscape and effectively recalling stored memories.

Hopfield networks align connection weights with state pairs to create stable patterns, using the Hebbian Learning Rule.

To store multiple patterns, the weights are summed, creating local minima in the energy landscape.

However, there is a limited capacity for storing patterns, approximately 0.

14 times the number of neurons, and similar or correlated patterns can cause interference, leading to unreliable recall.

Vanilla Hopfield networks, despite their limitations, offer a powerful model of associative memory through local learning and inference.

They have paved the way for more advanced energy-based models, such as Boltzmann machines, which introduce hidden units and stochastic dynamics.

A modern extension of Hopfield networks was published in 2016, with John Hopfield as a co-author.