Understanding Shaders in Game Development

Shaders are code running on GPU, defining how objects are rendered in games, using properties like position, textures, and colors.

### Understanding Shaders in Unity: A Practical Overview
* 🎨 Shaders are a fundamental part of game development, with everything in a game being **rendered with shaders**.
* A shader is essentially code running on your GPU, specifically focusing on shaders that **render graphics onto your screen**.
* 🎮 Shaders are like "**recipes**" that take input data, run calculations on it, and **output the final image** to your screen.
* 📊 Shaders can use various types of input data, such as **textures, colors, values, and mesh data**.
* 🔺 Vertex shaders operate on each vertex of a mesh, allowing you to **transform and manipulate** the geometry of an object.
* 🎨 Fragment shaders, also known as pixel shaders, determine the **color of each pixel on the screen**, controlling how the **final image looks**.
* 🛠 Unity shaders are structured using ShaderLab for defining properties and passes, and HLSL (High-Level Shading Language) for **writing the actual shader code**.
* 📚 Learning about shaders involves understanding the **basic structure of a shader file**, the role of **properties, subshaders, and passes**, and writing code for **vertex and fragment shaders**.

### Unity Shaders: A Comprehensive Guide
* 🎨 The vertex shader positions vertices in clip space, transforming them using an MVP matrix. It is commonly used for **per-vertex operations**, such as defining vertex positions or passing data to the fragment shader.
* 🔮 Interpolators pass data from the vertex shader to the fragment shader, **interpolating values across the surface**. They are defined in the vertex shader and **accessed in the fragment shader**.
* The fragment shader sets the color of each pixel. All shading and rendering logic is typically implemented in the fragment shader, making it a **crucial part of the shader pipeline**.
* 🎚 Shaders in Unity are structured using ShaderLab files, which contain **properties, subshaders, and passes**. Materials are used to configure shader properties and apply shaders to objects.
* 💡 Lower precision data types, such as "**half**" and "**fixed**," are often used in shaders to **improve performance** and **reduce memory usage**.
* 📦 The output of the fragment shader is the **color that gets written to the frame buffer**. The **SV_Target semantic** specifies the **output target**.
* 🌈 Colors and vectors are treated the same in shaders, as they both consist of **floating-point values with up to four components**.
* 🔄 Swizzling allows for **reordering and duplicating components** of a vector or color, providing a **convenient way to work with shader data**.

### Shaders: Understanding Vertex Shaders, Fragment Shaders, UV Coordinates, Normals, and Using Math to Create Effects
* 🎨 Shaders consist of vertex shaders and fragment shaders, where vertex shaders process vertices and fragment shaders **determine the final color** of fragments or pixels.
* 📐 UV coordinates are 2D coordinates on a mesh that determine how textures are applied, while normals are vectors that **define the direction perpendicular to a surface**.
* 🔄 It is generally more efficient to perform calculations in the vertex shader, as there are usually **more pixels to process than vertices**.
* 🎚 The `saturate` function clamps values between **0 and 1**, preventing colors from exceeding the **valid range**.
* 🔺 By using math functions like `abs` and `cosine`, it is possible to create effects such as **triangle waves** by manipulating UV coordinates.
* 🔌 Properties in shaders allow for **dynamic data that can be set per material**, enabling customization without changing the shader code.
* 📝 The structure of a shader includes the **shader path, properties, vertex and fragment shaders**, and any **additional functions or calculations**.

### Unity Shader Fundamentals: Vertex and Fragment Shaders, Blending, Depth Buffer, and GPU Parallel Processing
* 🔢 Unity shaders are composed of **vertex and fragment shaders**, with the vertex shader processing **per-vertex data** like mesh positions and the fragment shader rendering pixels based on **interpolated values**.
* 🎨 Shaders define properties for custom inputs, with sub-shaders used to **select between multiple shader variants** based on **performance characteristics or detail levels**.
* 🏷 Shader tags, such as **pass tags** and **subshader tags**, are used to configure shader behavior, with the distinction between them being **important to avoid confusion**.
* 💻 The CG program in shaders contains the HLSL code, with everything inside the CG scope considered **shader code**.
* The depth buffer is used to optimize rendering by **discarding fragments behind other geometry**, with shaders able to **control depth buffer read and write operations**.
* ➕ Additive blending combines the source and destination colors by **adding them together**, while multiplicative blending **multiplies them**.
* 📏 Vertex offset in shaders allows modifying vertex positions, leveraging the GPU's **parallel processing power** to efficiently **deform geometry**.
* 🌊 Vertex offset can be used to create effects like waves, with the shader code running for each vertex to **displace the mesh**.

### Shaders: Textures, Mipmaps, and Assignments
* 🖌 Shaders allow you to **sample textures**, with the default being a 2D texture called "**Main Text**".
* 🔍 Mipmaps are downscaled copies of the texture at different levels, taking up extra memory but improving performance by selecting the **appropriate level** based on the **partial derivatives** of the UV coordinates.
* 📝 Assignments will be provided based on the covered topics, with the **code and textures** from the session shared.
* 🔒 In Unity, enabling or disabling mipmaps and choosing the filtering mode can significantly impact **texture quality and performance**.
* 🌊 When sampling textures in shaders, you can specify a mip level using `tex2Dlod`, which is useful for **certain effects or optimizations**.
* 🛠 It's crucial to **experiment and have fun with shaders** to deepen your understanding.
* 📦 The code and assignments will be shared on Discord for those who want to **access them**.

Shader Basics, Blending & Textures • Shaders for Game Devs [Part 1]

Freya Holmér

Understanding Shaders: A Visual and Practical Approach

Shaders are essential in game development, as they are responsible for rendering graphics on the screen and can create a wide variety of effects, from glowing textures to refraction and distortion.

Shaders use parameters, values, and colors to perform mathematical operations and produce the final visual output.

They can also control blending modes, allowing for customization of how elements combine with the background.

In game development, shaders are versatile and ubiquitous, being used in different contexts with various inputs to achieve desired visual effects.

Using Shaders for Visual Effects in Games

Shaders are used to create specific visual effects on game meshes, such as magical textures and glowing animations.

They are also essential for rendering shadows, especially complex and dynamic ones, as well as for simulating the unique properties of skin, like subsurface scattering.

Additionally, shaders play a crucial role in providing detailed textures and geometry data through techniques like normal maps, offering a way to add intricacy to surfaces without increasing the polygon count.

Fresnel shaders are commonly used to create a glowing outline effect on objects, with the angle of surfaces affecting the intensity of the glow.

Shaders in gaming are not only used for realistic effects, but also for artistic purposes, as seen in games like Return of the Obra Dinn and Kentucky Route Zero.

Specialized shaders are employed in gaming for specific purposes, such as the portal locomotion system in Budget Cuts and the VFX shader for the rocket flame in Flowstorm.

High-end games like Uncharted demonstrate the impressive capabilities of shaders in creating realistic water surfaces and other visual effects.

Creating a VR player camera involves rendering two sets of cameras, one for the head and another for the teleportation beacon, using a depth buffer hackery technique.

The decision to code a shader versus using node-based editors depends on the render pipeline being used, with the option to involve a team of artists for shader creation.

Different methods for generating maps, such as normal and displacement maps, include sculpting, 3D scanning, and using tools like Substance Designer and Substance Painter.

A shader in Unity consists of a subshader, which contains one or more passes, with the actual shader code written in HLSL.

The vertex shader operates on each vertex of a mesh, primarily used for positioning vertices in clip space and for per-vertex operations, while the fragment shader iterates over each pixel or fragment to determine the final color of the rendered object.

The communication between the vertex shader and the fragment shader is one-way, with the vertex shader primarily setting the vertex position and passing data to the fragment shader.

Shaders in Unity are used to render objects and are configured using properties and materials.

Materials contain explicit values for the input parameters of a shader and have a reference to the shader.

Multiple materials can use the same shader with different input data.

The structure of a shader in Unity includes properties for input data, subshaders containing passes for rendering, and shader code for the vertex and fragment shaders.

cg inc for unity-specific functions, defining properties, and variables for the shader's input data, and using mesh data for per-vertex information, such as vertex position and UV coordinates.

Understanding Mesh Data and Interpolators in Vertex Shaders

Mesh data, including vertex position, UV coordinates, normals, and tangent direction, is crucial for shading and texture application.

Multiple UV channels can be used for different types of textures, and meshes can include float four UV channels for procedural generation.

Interpolators in the vertex shader are used to pass data to the fragment shader, allowing for the smooth blending of values, such as normals and vertex colors, across the surface of the mesh.

The interpolated data is calculated for each fragment rendered, providing a seamless transition of values across the triangles of the mesh.

Understanding Shaders: Vertex and Fragment Shaders

Shaders in Unity automatically handle the passing of vertex data into the mesh data structure, so there's no need to deal with that manually.

The vertex shader is responsible for returning interpolators, which specify the data to be interpolated across the surface, and it also handles the conversion of local space to clip space using the Unity object to clip position function.

In shaders, there are different precision data types available, including float, half, and fixed, each with its own level of precision, and they are generally used for different purposes based on the required precision and the platform of deployment.

Using lower precision in shaders can lead to faster calculations and, in some cases, reduced memory usage, which is beneficial for performance, especially on lower-end platforms.

It's generally recommended to use lower precision when possible, but it's important to be cautious and only optimize precision when necessary to avoid potential issues.

Understanding Shaders: The Basics of Color and Vector Manipulation

Shaders use the rgba structure to represent color, with the alpha channel commonly used for transparency.

Swizzling is a useful technique in shaders for reordering and accessing vector components, allowing for flexibility in color manipulation.

Shaders can be used to output different colors, and while it's challenging to automatically put a value in the alpha channel, there are ongoing discussions about this in the shader community.

Using Color and Normals in Shader Programming

Color properties can be set in materials and accessed in both the vertex and fragment shaders, allowing for dynamic color configuration.

By passing normals from the vertex shader to the fragment shader, the direction of the normal for each pixel can be visualized, with the option to convert it from object space to world space.

Shaders are designed to never crash, allowing for the output of negative color values, and the rendering of values above one, with different effects based on where they are outputted.

Unity's Matrix Multiplication and UV Manipulation

Matrix multiplication in Unity's include file allows for manual space transformation, specifically from object to world space, with the option to use the Unity matrix m for the same purpose.

Shifting the focus to shader optimization, it is recommended to perform as much as possible in the vertex shader rather than the fragment shader, especially when dealing with normals and other mesh data.

UV coordinates in Unity, representing 2D coordinates on a mesh, can be easily visualized and manipulated for scaling and offsetting in the vertex shader, providing a powerful tool for creating gradients and visual effects.

Creating a Simple Color Gradient in Unity Shader

By using the lerp function, it's possible to blend between two colors based on the x-coordinate of the UVs in shader code.

The inverse lerp function can be used to change where the gradient starts and ends by remapping the x-coordinate of the UVs to a different range.

To ensure the gradient doesn't overshoot into other colors, the use of the frac function and saturate to clamp the values between 0 and 1 is crucial in shader code.

Saturate is a built-in function in the shader language and is not part of Unity's include.

The function smooth step in shaders involves a lot more than just an inverse lerp and includes a cubic smoothing of the function.

Using math functions in shaders allows for the creation of effects and animations, such as generating a triangle wave, and the use of trigonometric functions for repeating patterns.

Unity shaders are primarily defined by properties, sub shaders, and tags, with the option to have multiple sub shaders for different performance needs.

The CG program, equivalent to HLSL code, contains the shader code, while the vertex shader processes mesh data and the fragment shader renders pixels.

The shaders allow for easy manipulation of wave patterns and can be adapted to work in different coordinate spaces for various effects.

Using shader code, it's possible to create a diagonal wave pattern by offsetting the coordinates based on the vertical position and applying trigonometry.

The shader can also be animated over time by accessing the built-in time variable and using different components for varying speeds.

Additionally, the shader allows for the creation of a fading effect and can be manipulated for additive blending to create flashy visual effects.

Additive blending is achieved by setting the formula to "source * 1 + destination * 1", and multiplicative blending is achieved by setting the formula to "source * destination + destination * 0".

When dealing with transparent objects, it's important to consider the depth buffer and whether to write to it, as transparent objects should not write to the depth buffer to ensure proper rendering.

Rendering multiple transparent objects in front of each other can be heavy and lead to performance issues, especially with complex textures and effects, causing a high fill rate.

Setting the render type to transparent and the render queue to transparent in Unity's built-in render pipeline is crucial for defining the draw order of transparent objects and ensuring they are rendered correctly.

Understanding Render Type and Shader Effects in Unity

Render type in Unity informs the render pipeline of the object's type for post-processing purposes.

Setting the queue to transparent ensures that transparent objects are rendered after opaque ones to avoid depth buffer issues.

Shaders in Unity manipulate math, colors, and vectors to create effects, and knowing the math behind it is crucial for shader programming.

The vertex shader in Unity is powerful as it can change the position of vertices, and this operation is efficiently handled by the GPU in parallel, making it much more efficient than performing the same task on the CPU.

The GPU is powerful for processing parallel data, making it useful for tasks unrelated to graphics.

Offloading processing to the GPU is done by writing shader code or compute shaders and retrieving the data through buffers or textures.

It is possible to create a radial pattern in the fragment shader by using the distance to the center and applying it to the wave function.

Using the "2d" specification sets the texture as a 2D texture for sampling.

The "main text" variable in Unity contains the surface color information and is the default name for the main texture property.

The "tex2D" function is used to pick a color from the texture, with UV coordinates typically used for this purpose in a normalized coordinate space from 0 to 1.

Mapping Textures to World Coordinates in Unity Shader

World coordinates can be used to map textures in a top-down projection, replacing the need for UV coordinates.

This allows the texture to repeat in world space and is commonly used for terrain to avoid constantly updating UV coordinates.

By using world coordinates for texture mapping, it's possible to create stretching effects, which can be useful for flat surfaces like terrain.

This technique provides a flexible approach in shader development, allowing for the interchangeability of different color and coordinate mappings.

Mixing Textures and Understanding Mipmaps in Shaders

Textures can be sampled in different spaces and blended together using top-down projection in Unity.

Mipmaps are downsampled versions of textures stored in memory to improve performance and reduce aliasing when sampling textures at different distances.

Anisotropic filtering allows for a more precise texture sampling, especially at grazing angles, but it can consume more memory and potentially affect performance in games.

Texture filtering in GPU includes bilinear, point, and trilinear filtering, each with its own blending and anti-aliasing characteristics.

Mip levels are crucial in texture sampling for specific effects and are commonly used in lighting and image-based lighting for varying levels of detail.

When using textures in the vertex shader, it's important to note that the vertex shader cannot automatically figure out the mip level, requiring the use of the "tex2Dlod" function.