this video is a tutorial for beginners to learn opengl using the python language. We will analyze the basics and fundamental concepts, go from rendering a simple triangle to more complex scenes with custom models. In simple words, we will create our own 3D graphics engine step by step. So let's start with the fact that opengl is a specification that defines an application programming interface to a video card for creating applications that use 2D and 3D graphics. We will work with the so-called modern opengl. This concept arose in 2010 when the version of opengl 3.3 was presented, and from that moment, this API uses a programmable pipeline with the obligatory use of shaders. To work with modern opengl, I will use the modern gl module because when compared with the standard pi opengl module, it has a number of advantages that allow you to develop applications much faster and more conveniently. So let's start in practice to understand what opengl is. First, we need to install the pi game, modern gl, and numpy.

modules using the pip install command, and now let's import the pi game, modern gl, and system modules. Then write a class with the loud name graphics engine. Let the size of our window be the input parameter. Now we initialize the pygame modules, take the value of the window size into the attribute, and set the attributes for opengl. We need to specify which major and minor version of opengl we will use. Here we assign version 3 to 0.3. We also set an attribute for the profile mask. We use the name profile core, which means that the deprecated functionality will not be used. Then, using the set mode method, specifying the window size, we will create an opengl context. For this, we use the opengl and double buffer flags. Now we need to detect this context using the modern gl module. It will automatically detect the created context using pygame. We will also need an instance of the clock class to set the frame rate and delta time. Next, we need a method for checking events where we will monitor the event for closing the window and also pressing.

the escape key. And when such an event occurs, we will gracefully exit our application. Let's write the render method. Here we clear the frame buffer, that is, fill the screen with the selected color, and then swap the buffers using the flip method. It remains for us to write the run method to start our application. Here there will be a main loop in which we will call methods to check events, the render method, and set the frame rate to 60 frames per second. And then let's create an instance of our application and call the run method. Let's run the program and we can now observe the set resolution window, which is filled with the color we have chosen. Well, as you can see, creating an application class in an OpenGL context is quite simple, and now we have everything prepared to implement more interesting things. So now I propose to create a new file and call it model. And using the example of creating a triangle model, we will look at how the OpenGL pipeline works. Then let's create a class of the same name and let it have access to the

application instance into our opengl context, and let's take a look at the simplified rendering scheme of the opengl pipeline. The pipeline receives information about the vertices of the object. Then, for each of the vertices, the vertex shader is launched. Let's say that it processes the vertices and passes it to the primitive assembly stage. This assembly is carried out based on information about vertex connectivity. So, then comes the rasterization stage where the primitive is divided into fragments. And now, for each fragment, a fragment shader is launched, with which the color of the fragment is determined. Then, there are various tests after which the fragment becomes a pixel and output to the frame buffer. And as you will see later, to get the desired result, the vertex and fragment shaders have access to uniform variables that we define ourselves. Well, as you just saw, we first need to get the vertex data for our model. And for this, we will write the appropriate method where we define the coordinates of the triangle vertices, taking into account the fact that opengl uses a right-handed coordinate system.

and here it is important that the vertex data was a numpy array, and the data type was float32. So when we have vertex data, we need to send it to the GPU memory, and for this, we create a vertex buffer object for which we specify our vertex data. Now we have come to writing shaders. Then let's create a shaders folder, and it will contain files for vertex and fragment shaders. For them, we use the name default, but the extensions will be vert and frag, respectively. OpenGL uses its own GLSL language to write shaders. So let's go to the vertex shader. Here you need to specify the version used. In the main function will become the entry point. Here you must specify the gl_Position variable as a four-component vector, since the vertices must initially be specified in homogeneous coordinates. In the fragment shader, we will also specify the version and leave the main function empty, since we will return to shaders a little later. Now we can load our shaders using a separate method. Here we will use the context manager to load them safely.

but we loaded them. In order to create a shader program object, it is needed in order to compile our shaders using CPU, and they will be ready for use on the GPU side. Let's access the vertex buffer and the shader program object through our class constructor. [Music] And now we have come to another fundamental concept in OpenGL: the vertex array object, in which we just associate our vertex buffer with the shader program. Please note that for the buffer, we specify the data format and attribute names. In our case, this means that in the buffer, each vertex is assigned three float numbers, and this group of numbers corresponds to an input attribute called in position. Okay, then create a vertex array in the class constructor and move on to the vertex shader. So we now have the coordinates of the vertex as a three-component vector in position as input, and its attribute location number is equal to zero. We pass it to the gl position variable for the further rasterization process. In the fragment shader, we define that the fragment color will be determined by

the frag color variable. And here we define the location number of our frame buffer. And now let all the fragments be read to render our model. We need to call the render method on the vertex array object. And here we must not forget to write a method to remove all created resources since there is no garbage collector in OpenGL. Well, now let's import the triangle model, create an instance of its class, and call the render method. And when the application is closed, we will release the resources with the destroy method. And as you can see, our model is displayed in red. So we can say that we got the Hello Triangle achievement. But the triangle is not a very interesting model. And let's now create a cube model. If we place the cube in the center of the local coordinate system, then its vertices can be assigned coordinates. And in addition, these vertices can be numbered. And then you can very simply describe all the triangles that make up the cube. There will be 12 of them. Let's use this information and describe first the vertices of the cube and then

using the numbering of the vertices, we will describe all the triangles that make up the cube. And by the way, in OpenGL, by default, the order of describing vertices is counterclockwise. And for convenience, we will write a static method for generating vertex data based on vertices in their indices. Then we use this method to get the vertex data and use the cube model instead of the triangle. And if we run the program, we will see something strange. Our entire screen is filled with a red rectangle, but in fact, this is a cube. We simply did not take into account that the model itself must go through its own transformation pipeline. That is, our model must first get from the model space to the world space, then to the camera space, and then to the clip space. All these transformations are done using the model view and projection matrices. Well, then I suggest creating a new file called camera and install the handy pi glm module, which is designed to work with OpenGL math. Let's import this module and first set the parameters for vue frustum.

now let's write the camera class. In the constructor, we calculate the value of the aspect ratio of the screen. Then we have all the data to form the perspective projection matrix using the glm module. To access this projection matrix, let's import the camera class in the main file and create an instance of its class. Now we can return to the model file. For clarity, we will expand the constructor with the on init method. In this method, we will pass the projection matrix from the camera instance to the shader. So, in the vertex shader, this will be a uniform variable of a 4x4 matrix. The positions of the vertices must be multiplied by this matrix. If we run the program, we will see the same picture. But this is because our camera is inside the cube. Let's move our cube back a little to make sure that we are really working with the cube model. But for now, we only see one side. Let's leave the cube in its place and move the camera itself. For this, we select the position for the camera and

the so-called up vector to form the view matrix. We again use the glm module in the lookAt function. Here we need to specify the position of the camera, the direction in which it looks, and the up vector. Such a function is very useful and saves us from extra calculations to get the view matrix. And now the view matrix can be passed to the input of the shader. It will be the same uniform variable that we multiply by the projection matrix. And if we look at the intermediate result, we can say that we really have the outlines of the cube, since the camera looks at its center and is located higher on the right. But our transformations are still incomplete, since we need to get the model matrix. We will get it using a separate method. And now let it be a four by four identity matrix. Then we pass it to the vertex shader in the same way, and it is multiplied by the view matrix. Since this is an identity matrix, the result in rendering the cube has not changed in any way yet. But let's still use the model matrix as

an example for this. We create an attribute for tracking time in the main file. Create a method for getting the time in seconds and call it in the main application loop. Next, I suggest calling the update method before rendering the model, and in this method, using the rotate function of the glm module based on the time value, we will rotate our cube around the y-axis. But then, at each iteration, we need to update the model matrix in the shader. Let's look at the result, and as you can see, our cube model began to rotate. But since the cube is filled with one color, this does not allow us to feel the full depth of the object. In that case, I suggest moving on to texturing our cube. Let's go back to the method of getting vertex data. Let's take a look at the test texture. So the texture coordinates are in the range from zero to one along the x and y axes. And for our case, for the convenience of working with them, they can be numbered. And if we look at the front face of the cube, it is easy to understand how to

apply texture coordinates to it. That is, with the help of texture coordinate numbers, we can describe all the triangles in the cube model. And then you can do the same as we did to describe the geometry of the cube. First, we define a list of texture coordinates, and then using their indices, we describe all 12 triangles from which the cube model is formed. At the end, we use the numpy module to combine the geometry data and texture coordinates data into one array. And since our vertex buffer has changed, we need to change the buffer format in the vertex array object. Specify that the first two numbers are texture coordinates, and then define the name of the attribute for these coordinates. So, in the vertex shader, we indicate that we have a new attribute in tech scored. We also define an output variable already for the rasterized texture coordinates for the fragment shader and let them be called uv. Well, the fragment shader now has access to uv coordinates. And since they are in the range from zero to one, let's display them in red and green color components.

and now we are seeing a rather strange picture. The faces in the cube are not ordered in any way. Since we need to activate depth testing to do this, let's go to the main file. In the application class constructor for the OpenGL context, set the depth test flag. And as a result, we can see the correct rendering of our cube. Along with the depth test, we can activate the call face flag, which does not render invisible faces. The rendering of the faces is affected by how we set the geometry of the vertices for triangles. By default, the vertices are set counterclockwise. But if we change the mode to clockwise, we will see the internal faces of the cube. So let's get back to our texturing, and we're almost done. And of course, we now need to load the texture itself. Here we will use the Pygame module. That is, we will write a method that loads the texture at the specified path. And then we create a texture object already on the GPU side. For this, we need to specify the texture size, the number of color

components and convert the texture to a string. So, to move on, create a textures folder and place the textures you need there. Then, we use our method to load the texture at the specified path. And then, go to the on init method. And here, we need to specify the name of the texture variable and the number of the texture unit and call the use method for our texture. So, in the fragment shader, we get access to the texture. In GLSL, this is the sampler2D data type. And using the texture function, we get the desired color for the fragment using UV coordinates. And if we run the program, we will see our textured cube. But upon closer inspection, we see that the texture mapping is reversed along the y-axis. And we did not get the desired result. This is due to the fact that in Pygame, the y-axis is downward. So, we need to use the flip method and flip the texture along the y-axis. And so, we have a correctly textured cube. And so, let's change the texture to something more interesting. By the way, let's reduce the rotation.

speed of the cube, and I will load the texture of the old wooden crate. Well, I guess our cube looks much better, but at this stage, I suggest giving us freedom of movement. So, let's move on to our camera class. So, OpenGL uses a right-handed coordinate system, and the movement of the camera can be set using the corresponding vectors: up, right, and forward. That is, we will simply move it along these vectors. Then we set these vectors as unit and orthogonal. Let's define a variable for the speed of movement of the camera, and we will change the position of the camera using the move method. We will track the keys pressed using the Pygame module. And in order for our movement to be independent of the frame rate, we need a variable for delta time. The delta time value itself is easy to get using the clock instance in the main application loop. And then we can start writing the move method here, taking into account delta time. We will get the velocity value, and using the Pygame module, we will get information about the keys pressed.

and then we will change the position of the camera by moving it along the forward writing up vectors in accordance with the selected keys for moving the camera. And we will call the move method through the update method. And since the camera position changes, we need to recalculate the view matrix. So we will call the update method in the main loop before rendering. And since the view matrix has also changed, we also need to pass a new view matrix to the shader. But if we run the program, we will see that we have a rather strange control. This is due to the fact that the orientation of our camera is always directed to the center of the cube. Then when calculating the view matrix, let's set the direction of the camera from its position taking into account the forward vector. And in this way, we get the opportunity to move in space. But as you can see, it is impossible to change the camera orientation now. So let's solve this issue. So there is such a thing as euler angles with which you can describe any rotation in space. As you can see from the slide.

this is pitch, roll, and yaw. And for our engine, we will use pitch and yaw. Let's then, for convenience, the values of position, yaw, and pitch come to the input of the constructor, take them into attributes, and define a variable for mouse sensitivity. Since we will change the camera orientation with the mouse, now you can write the rotate method. Here we use the data on the relative movement of the mouse for the previous frame, and based on them in the sensitivity, we calculate new values for yaw and pitch while limiting the pitch within so that there are no unnatural turns up and down. And most importantly, we need to write a method for recalculating camera vectors taking into account yaw and pitch. The forward vector is responsible for the orientation of the camera, so using basic knowledge of geometry and trigonometry, you can find the projections of this vector on all world axes. And when we have calculated all the components of the forward vector, then using linear algebra in the cross product of vectors, we will first calculate the right vector using the fake up vector, and then find the true up.

vector and do not forget to normalize each of the vectors. And it remains for us to call our rotate methods in the vector recalculation method through the update method. Thus, we got the classic implementation of the first-person camera for our engine. But we can make one more improvement so that the mouse pointer does not go beyond the window. To do this using pygame, you can use the event set grab method and also hide the mouse pointer itself. Well, now we have a very convenient control and besides complete freedom of movement. But let's go back to the moment when the object is painted in one color. Then you cannot use the shader and just fill the texture with the specified color. And we are again observing our object without any sense of its depth. And we have come to the point where we begin to implement lighting for our engine. Then let's create a file called light. Here we will write a class with the same name. The attributes of which will be the position of the light source in its own color. Looking ahead, I will be implementing a

fong lighting model. So, we will set the light intensities for the ambient, diffuse, and specular components. Let's import this class in the main file and create an instance of it. And now we can pass the intensity of the ambient component to the shader. To begin with, but we do this taking into account the fact that in the fragment shader for convenience, we create a structure called light. The elements of which will correspond to the attributes of our instance. And then we can declare the uniform variable light. We will calculate the lighting using the get light function. In which for now, we will multiply the color obtained from the texture by the value of the ambient component. And let's apply this function. And now we can see what the ambient lighting looks like. But in order to move on to calculating the rest of the lighting components, we need to return to the method for obtaining vertex data and talk about normals. So, in simple terms, normals are vectors perpendicular to surfaces. And in our case, for six faces, we have six normals. And since our face consists of two

triangles. Each triangle consists of three vertices. Then, we make a list so that for every six vertices there is the same normal. Let's transform this list into a numpy array and again combine it all into a single array of vertex data. And accordingly, in the vertex array object, we change the format for the vertex buffer and add a new attribute in normal. Let's make a change to the vertex shader that we have a new attribute and declare the output variable normal so the lighting calculation is performed in the world space. And it would seem that it would be enough to multiply the normal vector by the model matrix without taking into account translation, but not everything is so simple. And in order for our lighting to be correct when the model is not uniformly scaled, we use more complex mathematics. First, invert the model matrix and then transpose. And besides, for lighting, we need to calculate the position of the fragment also in world space. So in the fragment shader, we have input variables for the fragment's normal and position. So let's go back to the get light function and normalize the

components of the normal vector. To determine the diffuse component, we will use Lambert's Law. According to which, the illumination of a surface area is directly proportional to the cosine of the angle between the normal to the surface and the direction of light. And in turn, the cosine of the angle is equal to the dot product of these vectors. Then, we will use this information to calculate the direction to the light source. And taking into account the normal, we will find the value of the dot product of these vectors. Do not forget to use the diffuse light intensity value and calculate the final lighting color for the fragment. And it remains for us to pass in the on init method to the shader the values of the position of the light source and the intensity of the diffuse component. And if we look at the result, then we can say that our cube finally looks like a three-dimensional object. And as you can see, lighting plays a very important role in the perception of objects. So, for the full lighting model, we need to calculate the specular component. In this case, we need two vectors, the first

vector is the direction to the camera, and the second is the vector reflected from the direction of the light source. Then the dot product of these vectors raised to some power will give us a specular highlight. Let's then calculate the vectors we need and find their dot product, and let's say we raise it to the power of 32. Take into account the specular intensity and calculate the final color of the fragment. It remains for us to pass the specular intensity to the shader, and we also need to pass information about the camera position. But since the camera is not static, we will pass the position value in the update method. And also do not forget to declare a uniform variable for the camera position in the shader itself. And as a result, we can see the specular component on the upper surface of the cube. The value of the number to which we raise the specular component determines the strength of shining. The larger this value, the stronger the light will be reflected. So if we return the previous look of the texture, then we can say the Phong

lighting model is ready. And by the way, we used a point light source without attenuation here. You can easily find information on the implementation of other light sources on the internet. And now I would like to talk about the current structure of the project. As you can see, the model class is overflowing with various methods. Perhaps this approach is suitable if you work with a single instance, but when creating several instances of this class, it will lead to an unreasonable consumption of computing resources in memory. Since for each model, we'll load the same shaders, textures, the same vertex buffers will also be created, and so on. And in the simplest case, for this, we need to reorganize the project approximately to the following class architecture where we separately have access to shader program objects and various vertex buffers based on them. We can create the vertex array objects we need, and in a separate class, there is access to textures. And for convenience, all this is aggregated in the mesh class, with the help of which we will already more efficiently create object instances. So now our project is presented with a

new file and class structure. According to the slides, let's take a quick look at it. A class has been created in the shader program file where we will use the previously used method to load the shaders we need. Create an object of the shader program, and we will access such objects through a dictionary called programs. There will also be a separate method for releasing resources. Next comes the file for vertex buffer objects. Here, for the convenience of further work, a base class with three methods familiar to us is created, and already inheriting from this class, the cube VBO class is created. In the constructor of which, we define the data format for the attributes and the names of the attributes themselves. And here we already have our method for getting our cube's vertex data. There is also a main class in which through a dictionary, we will access the VBO we need. So in the Val file, we import and create instances of the shader program in VBO classes. And using the get Val method, we create the vertex array object we need, which can also be accessed through a

dictionary named valves. Next comes the texture file. Here the class of the same name is arranged according to the same principle. We load textures using the appropriate method and access them through a dictionary called textures. Here, for example, three different textures are loaded. So the file with the mesh class. At the moment, it is very simple. We import the vowel and texture classes and create their instances with the destroy method. And we import the mesh class itself in the main file and create an instance of it in the graphics engine class constructor. And as for the model file, here I suggest using the base class for the model, which specifies the name of the vowel and the texture number. And then accordingly, through the mesh instance, we get access to the desired vowel and through it to the shader program. Besides, here we leave three basic methods. And then to create a cube class, it is enough to inherit from the base model class, specify the vowel name and texture number. And then we leave everything as it was for the update and on init methods. And if we run the program, then we can

still see our cube. We reorganized our project in order to be able to conveniently and efficiently create a large number of objects for our scene. And if we are talking about the scene, then let's create a file with this name. Here we will import the model and write the scene class with access to the application. In the constructor, we will declare a list of scene objects. In a loading method, we will write a method for adding an object to the list of objects. Then, in the load method, we will place our cube in the scene. And in the render method iterating over the objects also called the render method. Let's check the work of this class, import it in the main file, and instead of a cube, create an instance of the scene. As you can see, everything is working correctly and we have come to how we can add more objects to the scene. And for this, we need to talk about transformations. As you probably understand, in order to place an object in space, it needs to be given the appropriate coordinates. And for this

will calculate the model matrix, taking into account this position. We use the translate function of the glm module. And in the class for our cube, we will also specify the input coordinates for its position. Then let's place two more cubes in our scene on opposite sides of our cube, but only with different textures. And as you can see, we got a way to place our objects in space. But in addition to this, the object can be given its initial rotation around the axis. So let's set the rotation around each of the axes in Euler angles, and for convenience in degrees. And then first convert them to radians. And then we use the rotate function three times separately for each of the x, y, z axes. And also provide the cube class with the ability to set the initial rotation. Let's give the cubes a rotation of 45 degrees around the x and z axis, that is the pitch and roll values. So now we are able to set the initial rotation of all objects, but there remains one more fundamental transformation: scaling. You can also scale an object along three

axes. We will similarly use the scale function of the glm module, and it is important to note that the correct order of transformations is scaling, rotation, and translation. It is important to know that matrices are multiplied from right to left. Then, for example, let's stretch one cube along the y-axis and the other along the z-axis. Thus, we have the opportunity in our engine to apply basic transformations to objects. Okay, but to talk about some more nuances, let's add even more objects to our scene. Suppose we place a certain number of cubes in the xz plane so that we are in the center. So, we got a more interesting scene. And I made it to talk about gamma correction. Let's assume that the x-axis is our calculated color values and the y-axis is the color value displayed on our monitor. Everything is fine when we have a linear relationship, that is, what we calculated is what we got on the screen. But due to the peculiarities of color perception by our eyes and besides historically, it turned out that monitors

use a power law when displaying colors. Approximately the degree is 2.2. And then it turns out that a color value of 0.5 is displayed with a lower brightness, a value of 0.218. And in order to make the resulting colors linear again, we need to multiply the displayed color values in linear space by the inverse gamma curve, that is, make them brighter. So let's go back to the fragment shader and declare the gamma value. And then we can apply the gamma correction to the final color value. But if we run the program, we will see a completely unexpected result. Everything has become too overexposed. This happened due to the fact that we take the initial color from the texture, but the colors of the texture, as a rule, are already in nonlinear space. And therefore it turns out that the gamma correction actually applied twice to texture colors. And then it is enough for us to return the texture colors to linear space and look at the result. And thus we get the colors of objects that are correct in terms of gamma.

eye perception. [Music] So, if we make a solid floor of cubes in the scene, we will see ugly ripples and artifacts going into the distance. This is due to the use of one high-resolution texture for small objects. To solve this problem, OpenGL uses a technology called mipmaps, which provides a set of texture images where each subsequent texture is half the size of the previous one. To activate this technology, we go to the get textures method and first we specify the minification and magnification filter for the texture, and then we use the build mipmaps method. So now we do not see that ripple, but already strongly smooth textures go into the distance. And here we can additionally apply one more technique. And to improve the quality of the textures, we will use anisotropic filtering. Such filtering allows you to eliminate aliasing on various surfaces, but at the same time introduces less blur and therefore allows you to save image detail. So, in the next step, let's figure out how to use custom 3D models in our engine.

for example, in the classic obj format. And for this, we will find some free model and upload it. Let's put the downloaded files in a folder named objects and start with the vbo file. For the obj format, you can write your own parser, but for convenience, I will use the module pi wavefront. Install it using the pip install command and import it accordingly. So, for our object, we will write a class called CatVBO. Here we inherit from the base vbo class and we need to define a method for getting vertex data. Then we use the installed pi wavefront module with cache and parse parameters. After that, our model may consist of several objects stored in the dictionary, but in this case, there is only one object. Then we get access to the vertex data and convert it to a numpy array. And if we look at the data format of this model, we will see that they completely coincide with the previously defined format and no additional changes are required. And in order to have access to this vertex buffer, then we will place an instance of our CatVBO class in the

vbo's dictionary. Now, we can create a vertex array object for our object and put it in the vowel's dictionary. And since the shader program will be the same, we simply use our created vbo. Also, in the texture dictionary, we must have access to the texture of this model, load it with the appropriate path, and now we can make a cat model class. For now, we will make it based on the cube model. And here, we need to specify the name of the vowel and texture. Let's go back to the scene class and add an instance of our model and place it away from the camera. And as a result, we can observe our model. But we see that its initial orientation does not fit our coordinate system. Then you can easily change the initial orientation of the model. Let's rotate it 90 degrees. And in addition, let's add our cubes to the scene. Well, in this case, we can observe our model, which is made in obj format. You can use different formats of 3D models. And for this, there are various modules for their convenient loading.

so we have covered only the most fundamental and basic concepts for creating applications with Python and OpenGL. But I also do not exclude the release of further series on learning OpenGL. [Music]

Let's code 3D Engine in Python. OpenGL Pygame Tutorial

Let's code 3D Engine in Python. OpenGL Pygame Tutorial

스크립트