In the previous part of the lecture, we discussed some basic aspects of machine learning and what topics we will cover in this course. Now, let's take a look at some mathematical concepts. There are a few mathematical areas you need to understand in order to grasp machine learning, including linear algebra, calculus, and probability. Let's focus on probability for a moment. This is important not only for solving various machine learning problems but also for understanding how to formulate these problems. Nowadays, we often use models like ChatGPT and diffusion models. Ultimately, the term token refers to the words we use; if you're not familiar with the term, you can simply think of it as words. These words are arranged in a sequence.

When there are thousands of options, it's a program that predicts what the next word will be. The way it predicts the next word is by estimating the probability distribution of that word. So, when our ChatGPT is generating text, there is actually a probability distribution that comes out for each token. For this token, the possible probability distribution looks like this; for the next token, the possible probability distribution looks like that. If there are, for example, 50,000 candidates for tokens, then there will be 50,000 numbers coming out, summing up to 1. If the possible tokens are 200,000, then there will be 200,000 numbers coming out. In this way, if we look at what neural networks essentially do, they often model probability distributions. In fact, there are quite a few neural networks that do this more often than not.

We are modeling probability distributions. In that aspect, the subject of probability is very important, and the unit on probability is crucial. When we want to solve a problem, we can set the unknown as a variable, but we can also treat the unknown as a random variable. We call it a random variable in English, and in our language, we translate it as 확률 변수. There are quite a few cases where we use random variables to solve problems, and it is one of the methods that works very well. Let's go ahead and start. This is not a very easy topic, but I have structured the course to introduce the new part. It would be worthwhile to listen to what we are aiming to cover in this subject during the introduction. It may get a bit more challenging later on.

There may be random numbers and random vectors. When we have a single number, we call it a scalar. If we have a random number, it is naturally called a random number. If there are multiple numbers and they are arranged in one dimension, we refer to it as a random vector. If this extends infinitely in time, the time index can be continuous or discrete, such as one, two, and so on. In that case, we can refer to it as a random signal or a random process. This is how we start, and if there are multiple numbers, it becomes a random vector. If there are infinitely many corresponding to time, we call it a random process. In machine learning, it is certainly used, but random vectors are the most common.

When we try to predict something, the state is often represented as a vector. You might feel that representing it with just a single number is insufficient. There are also problems expressed as random processes, but random vectors are most commonly used to represent something with a finite length. To properly understand the subject of random processes, various foundational knowledge is required. First, you need to understand sets. You might wonder why sets are suddenly relevant, but you'll get to know this in Chapter 1. Next, you should have some knowledge of counting methods, and of course, linear algebra is always fundamental. Although I may be a bit strict about this, having a good grasp of these concepts is beneficial. We will conduct experiments using Python programming, simulating different situations to see how they work.

This is what it is. Now, we are conducting experiments through computer programs. If you proceed, you will understand what that means. So, while pretty likely is mentioned, let's think about it carefully. Unfortunately, since this is a video lecture, I can't ask you questions, which is a bit disappointing. In high school, we have a subject called probability and statistics. In probability, we clearly learned about Gaussian distributions and probability distributions. Even if it's not the exact terminology, we learn about words related to probability and random variables. What this means is that those of you who chose this subject in high school, or even those who have been out of school for a longer time, must have taken a required course related to probability.

So, what exactly is probability? When we talk about probability, there's an even more important keyword, which is random variable. What is that? That's the first question. For example, what is a random variable? This is a question I keep asking. At the beginning of this course, during the online class, I couldn't ask questions, so I had to ask myself and answer myself. When we talk about random variables, we have the mathematical aspect and the real-world aspect. In the real world, it feels like whenever you just poke at something, a new number pops out. That something that pops out when I take an action is what we call a random variable. When I take some action...

What are some examples of new numbers popping up when we take action? The simplest example is the result of rolling a die; the number that comes up is what we're talking about here. So, every time I take some action, a new number pops up. This is an example of a random variable in the real world. Now, let’s think of more examples of random variables. For instance, when I stop a passerby, the height of that person is also a random number that pops up for us. Similarly, when I'm driving, the four-digit number on the license plate of the car in front of me is also random. So, every time I take an action, a new random number pops up; that’s what a random variable is.

So, what I'm saying is that when we flip a coin, the outcomes of heads and tails are not random variables. However, if we assign heads as 0 and tails as 1, then it becomes a random variable. Random variables are something we encounter a lot in the real world. Common examples include coin flips and dice rolls. Additionally, when we draw cards from a deck, if we assign numbers 1 to 13 to the cards and also assign suits as 0, 1, 2, 3, and 4 for clubs, hearts, diamonds, and spades respectively, then when I draw a card, I can see which number comes up. These are all examples of random variables. A random variable refers to...

There are things like boxes, and when I tap here, numbers pop out. When I tap, numbers pop out; it's the feeling of something that, when tapped, produces random numbers. This is meant to model the things we see in our eyes. Now, what is the mathematical counterpart to this concept of the real world? Here, it becomes quite abstract. In sample space, we define it as a function that goes to real numbers. You need to accept these parts a bit. When we model engineering problems in the world we live in, this object that produces random numbers every time it's tapped is very well modeled as a random variable. However, expressing these somewhat unclear concepts in the mathematical world can be challenging.

It's difficult, and creating a concrete theory based on that is also challenging. In the world of mathematics, this is defined in a very formal language. However, that formal language may not resonate well with us. In a sample space, a function that maps to real numbers is called a random variable. The important thing is that when we observe a phenomenon in a certain world, we might think, Ah, this can be modeled as a random variable. So we model it as a random variable, then mathematically formulate a method to solve it, and finally, we interpret the results. Therefore, when you need to create a program or system, there might be some external elements, but from our perspective, random numbers come in sporadically. In that case, modeling it as a variable can be very effective.

That sounds good. It's very random variable modeling. So, when I look at the world, there are many problems we need to solve and things we need to predict. I think there are two choices. First, we can approach it by determining the value at time t. For example, we can calculate the position of a ball after t seconds by solving Newton's equations. This means treating the unknown object as a fixed number and finding a solution that way. Alternatively, we can approach the unknown by treating it as a random variable rather than a fixed number. This means looking at it as a fixed unknown at time t and solving it in that context.

There are cases where we essentially view something as a random number, even if we don't know it. The left side is something we've seen and are so familiar with that it's easy to think that the world operates entirely that way. However, the right side definitely exists too. Even if we don't learn it in textbooks, there is always a principle at play. For instance, in the quantum world, we cannot place objects like we do on the left side. This is where we solve the Schrödinger equation to perform calculations. Now, think about it: when we solve the Schrödinger equation, what do we get? We get a wave function, and the square of the wave function represents a probability distribution. This means that certain properties, like the position or momentum of a particle in the quantum world, are modeled as random variables. So, we have random variables.

When it comes to modeling, it’s impossible to say exactly how much it is; all that comes out is a probability distribution. In the left world, I think of it as a fixed value and solve for a specific number or quantity. On the right side, by modeling, we get a probability distribution. The thinking processes on the left and right are very different, and there is a significant difference in approaches. We have primarily dealt with the left side until now, but the right side is also very important. In the microscopic world, there are fundamentally random elements. Even if something is not fundamentally random, when we try to assess it in relation to some physical process, the mathematical formulas become extremely complicated. For example, when rolling dice...

There are various cases to consider, such as how much spin is applied, the height, and the pressure, which can affect where the landing will occur and whether it will land on the front or back side. We can calculate these factors, but it becomes impractical in terms of cost-effectiveness. Instead, we could use the outcomes of a coin toss as probabilistic variables and derive a probability distribution from them. For example, we can represent the outcomes as 1, 2, 3, 4, 5, or 6. This is a valid approach. 

If we can perfectly control the situation to achieve a completely accurate result, that would be ideal, but achieving such precision may come at a very high cost. While we might be able to manage it perfectly, the associated costs or other conditions may make it unfeasible.

It's impossible to do that. When we say right, it's okay to refer to it in a fundamental way. There are also those who are fundamentally on the right side, just like in the microscopic world. So, the topics we cover in this subject are these three areas. You might not be very familiar with them, but how does machine learning work? It learns from data. However, the data comes from sampling in the world. Depending on how you sample, the results will vary. You instinctively know that this is deeply related to statistics, right? If you want to process that statistically, then of course, probability and such will be fundamental. Even in high school, the subject was called Probability and Statistics. So, when you think about all these things, you'll realize that this subject is quite useful. It has its own utility, and machine learning...

It is quite useful for understanding. So, let's get started. There are these concepts like sets, subsets, and the empty set. A proper subset, is that what we call a proper subset? It just fits in, but it can't be the same. Then, if there is no intersection, we say they are disjoint. This is something you might have heard a lot. So, this is something obvious; you might think of it as content we learned in high school. 

As for subsets, you may have learned terms like finite and infinite. This is a very intuitive part, but there are two types of infinity: countable and uncountable. Countable refers to natural numbers, which can be put in one-to-one correspondence. So, typically, integers and rational numbers can be matched one-to-one with natural numbers. However, real numbers cannot be matched in that way.

It is called Uncountable. I have explained all the content here. Now, you might think that with this information, you have understood the iPage. However, I want to say that there is a level of understanding that goes beyond just agreeing with the statements. So what does understanding mean? It means considering why the author, the person who wrote this book or created this material, arranged it in this way. Why is it that after discussing probability and random variables, the first topic introduced is about sets, something we learned back in high school? It's like when we watch a Marvel movie; a story is developed step by step. So, how does it unfold, and what happens next?

The stories come together, and the conflicts are resolved. In the end, we will do something with probabilities and random variables, but first, we need to start with sets. Understanding these sets is crucial because it will connect to how probabilities and random variables relate to each other. It is important for you to grasp why these concepts are interconnected. This is obviously a true statement, and understanding how it connects to the theory is key. From this perspective, it is essential to follow along. This relates back to what I learned in high school, such as union, intersection, and complement. I didn't learn all the details, but I hope you get the idea. For example, when we talk about A minus B, we can express it as A - B or in other ways.

I didn't learn this in high school, but this is how it's expressed. There's nothing particularly special about this; it's something I learned in high school. If you take this part too lightly or consider it unimportant, you might miss a significant story. The explanation about sets is being provided well. Now, what does this have to do with probability? We will find that out; it should come up around the next page. So, you can think of it as a review of sets. This is about the commutative law, then there's the associative law, and the distributive law, along with De Morgan's laws. These principles hold true, and since the upper and lower parts look similar, it's called duality. Up to this point, we have learned everything about sets. So, we are going to revisit the content about sets that we studied in high school.

I think you can consider this. I reviewed it. Now, the question is why this topic has come up. I didn't create this myself, but I found this kind of picture somewhere. The left side represents the real world, and the right side represents the mathematical world. The picture above was taken from my university's calculus book; it's very early in the book. It's a very natural and obvious statement, but we often overlook it. There are people who solve problems in the mathematical world, but there are also problems in the real world. So, how do we approach this? The most fundamental way to solve this is to convert the real-world problem into a mathematical problem. In other words, transforming the real-world issue into a mathematical problem is what we call formulation.

It has come out, but this is like modeling. We want to solve problems that exist in the real world. We want to resolve these issues. Instead of using the word solve, I think resolve fits better here. So, we want to resolve problems in the real world. How do we do that? We convert it into a mathematical problem. You know, we do this in elementary school, not high school. For example, if there are a total of 24 legs combined and the number of heads adds up to 19, we might have eight of something. Then, we figure out how many chickens and how many cows there are. So, what we do is let the number of chickens be x and the number of cows be y. Then, we have x + y and 2x + 4y, and we convert it into a mathematical problem to solve. In the diagram in front of us, we jump like this.

However, the jump in the process of modeling is somewhat intuitive. We cannot prove everything logically, but we can explain many aspects logically. Yet, there are areas that cannot be proven. For example, if we have one apple, and then we have two apples, we can correspond that to the natural number 2. But if we consider one apple versus two apples, if one apple is somehow less than two apples in terms of quantity, that makes sense. However, there could be cases where we have one apple that looks a certain way and two apples that look different. This situation arises. So, when we express natural numbers using apples, if the apples are of similar size and quality, then it becomes a reasonable model.

This has passed, so it has passed well now. However, there are rotten apples, small ones, and very large ones, and soon the prices will all differ. If we simply talk about it in terms of quantity, then this model fails. We need to use a model that fits the situation, but the interpretation of that situation is up to us when choosing the model. The part about jumping off a cliff is something we need to judge. If we jump off the cliff and turn it into a math problem, then we have already prepared various methods to solve the math problem, so we can just solve it. What comes next? We interpret the results and relate them to the real world, so for instance, you might say I need three apples or there were a certain number of chickens. This is how it gets organized. So, we turn it into a math problem, solve it, and then relate it back to the real world.

When we retrieve something, as I mentioned, random variables yield different numbers each time in the real world. This makes random variables very useful for modeling such subjects. After that, we jump and extract the information, and then we bring it in this way. In the mathematical world, random variables and probabilities are expressed using sets. It starts with a set, so we had a set in front of us. What this means is that probability involves the entire set, known as the sample space. When we have a sample space, we look at how large a particular subset of that sample space is. Measuring the size of the subset in relation to the sample space is the mathematical definition of probability. Let's summarize this again. This is a concept that you all need to grasp, and it’s not that difficult of a concept either. First, you need to think about it.

There is the real world and there is the mathematical counterpart of the real world. We transform problems from the real world into mathematical problems, solve them, and then return to the real world to provide answers. This is a framework that we use extensively. In the modeling from the real world to the mathematical world, there is a certain degree of subjectivity involved. Not everything can be explained logically. For example, if we say that current flows through a conductor, we can represent the current as I and consider it as a real number. However, we also know that current is generated by the movement of electrons, and since electrons have a charge of 1.6 × 10 to the power of -19 coulombs, this value is indeed in coulombs. However, this value must be multiplied by some factor, and time is also involved in this context.

It may not be completely logical, but we can talk about it in this way. We shouldn't consider that as a mistake; rather, we should treat it as a tied value or something similar. However, we tend to simplify those matters. The reason is that if we consider everything and try to address those issues, the problems can become too complex, or while it might resemble reality to some extent, it could lead to situations where we can't solve problems due to realistic constraints. Alternatively, we may also realize that we might not fully understand reality. Therefore, when we solve a problem and convert it into mathematics to obtain an answer, we need to adjust that answer to a level that is appropriate and practical. The same goes for flipping a coin. When we flip a coin, we say heads is 1 and tails is 2, but if you look it up on Google, you'll find that...

There are quite a few cases like this, and if you look up front and back sides, you will see that the probabilities are not the same. Just think about it; since they are not perfectly symmetrical, one side might be slightly preferred. However, if I set the probability of heads to 1 and the probability of tails to 1, then I've already done some modeling. You might want to use a more sophisticated model, and that's perfectly fine. You could include the side, and you really need to consider the weight and shape of the front and back to ensure that the probabilities are not equal. This is your choice. So, the choice of modeling is always made, and in physics, modeling is taught directly from the textbook, which makes it a bit clearer. But in the case of probabilities, it can be a bit different.

It seems like we're discussing problems in the real world using dice. There are two parts to this: one is the modeling aspect, and the other is solving the mathematical problems. It's important to distinguish between these two to enhance understanding. What we are talking about now on this page is the mathematical aspect of probability. If you look closely, there are three main components: sample space, events, and probability. The sample space is simply a set, representing a collection of outcomes. An event is a subset of that sample space. Probability refers to the sizes of these subsets, where we assign numbers to indicate their sizes. So, we can create a model related to dice.

If we create a model, it is a very simple one, not a sophisticated model. However, it is a model that we use quite often. An event is a subset; it could be 1, 2, 3, or 6. If we consider even numbers, we might say it could be 3 as well. In this case, the total number of subsets is based on the six elements, so it becomes eight. These are the events we have, and for each of them, we assign numbers like 1/16 for one and 1 for another, and so on. This is the mathematical probability model. When you roll a die, we talk about the probability of landing on a specific number, but that involves a very sophisticated and formal mathematical framework.

There is a mathematical counterpart, and in that mathematical counterpart, there is something called a sample space, which contains subsets. It describes how large each of these subsets is.

[05-01] 확률 Crash Course

[05-01] 확률 Crash Course