Advanced Detail Mapping Technique for Game Development

Using texture arrays for blending multiple detail normal maps in character materials creates efficient and detailed surfaces in 3D environments.

### Advanced Detail Mapping with Texture Arrays
* 🎨 In this tutorial, the instructor demonstrates how to implement **advanced detail mapping** using **texture arrays** in game engines like Unreal Engine and Unity.
* 👚 The example scenario involves creating detailed normal maps for different materials like **denim, leather, linen, and felt**, and blending them onto a character model based on a **material mask**.
* 📦 The "wrong way" to do this involves **sampling multiple normal maps separately** and **blending them based on a material mask**, which can quickly become **computationally expensive**.
* 🛠 The "right way" to achieve this effect is by creating a **texture array**, which is essentially a **stack of textures combined into a single asset**.
* 🎨 In Unreal Engine, the instructor demonstrates creating a texture array and sampling it based on a material mask to **blend different detail normal maps** onto a character model.
* 👥 The tutorial emphasizes the importance of creating **texture arrays with consistent resolution and format** for all **normal maps** used in the array.
* 💻 In Unity, the instructor combines the individual normal maps into an atlas texture before converting it into a **texture array**.
* 📊 By using texture arrays, the shader becomes more efficient, with **fewer instructions and texture lookups** compared to the "wrong way" of **sampling multiple normal maps separately**.

Today, we're going to talk about advanced detail mapping. Let's get started. So last week, we talked about detail normal mapping, and I showed you this example of a rock, and we added this really nice tight detail to the rock that you can see here. So when we zoom in, even though the main textures lose their resolution and kind of break down, we still have detail because we're tiling this additional normal map texture and getting extra detail on the surface. And right at the end of that video, I talked about another case where what if we're making something like a character where instead of our mesh just being made out of one material like this rock is, if our character has an outfit on that has denim and linen and leather and multiple materials, the technique that we looked at last week only has the option of using one detail map here. So we just have a detail map for rock, but if we need to add additional materials, there's no way of using one detail map in one place and another detail map in another place. So that's

what we're going to take a look at today. I'm going to show you a technique for adding different detail depending on what material is placed where on your material sets. So the first thing that I want to do is show you the wrong way to do it, and then I'll get into the right way to do it. So the first thing that I did is I went ahead and I created four different detail normal maps. Let's take a look at those in Photoshop really quick. All right, so here we are in Photoshop, and you can see I have a detailed normal map here for denim. I have one for felt, one for leather, and one for linen. The objective of these detail maps is just to create some really tight surface detail of what happens really close up on these different materials. Here in the denim, you can see that there are all these different little thread loops in the normal map. In the leather, you can see that we've got this nice skin texture going on, and then in the linen, we have some

thread as well. So the idea of these is when you get up really close on your character and you want to take a look, uh, at their clothing or, or what their, um, what the various materials are made of, that the textures don't break down when you get up that close, but you actually start to see the details that you would in real life on a character like this. And so we need a way of blending these four different detail, uh, textures together and then applying them to the base textures as well. Okay, so let's switch to Unreal, and I'll show you that. Like I said, I'm going to show you the wrong way to do it. So first of all, I've created this mask texture where one material is red, another is green, another is blue, and then the other one is in the alpha channel. So here's our alpha channel. You can see my mask is white in the alpha channel in this area. So I basically have just four different masks. And you can see I've set my compression setting to masks because that's what they are.

so i've got my red, green, and blue, red, green, blue, and alpha masks here. And then I brought in my normal maps: my denim, felt, leather, linen. And then let's take a look at the material that I created. Okay, so here's my material, and this looks a little bit crazy, and it is, and you definitely shouldn't do it this way. This is the example of how not to do it. So I've got my four normal maps here. They each have their own different texture sample. So I'm sampling all of them, and I have a multiplier here for how often they should be tiled. So for example, I could set this to 8 and tile them really tightly. And if we look on the preview over here, you can see that this detail shows up when we zoom in really close. But just for now, I'm going to set this to one. Normally, you'd want this value to be kind of high so that the detail normals are really tight on the surface. But just so we can see a little bit

better what's going on in the preview. I'm going to set that value to 1. So I'm tiling my texture coordinates. I'm sampling all four of my normal maps. Then I multiply each of them by the mask. So this one's multiplied by red, the second one is multiplied by green, the third one is multiplied by blue, and the fourth one is multiplied by the alpha channel. Then I add the results together, and I use the blend angle correct normals that we talked about previously. And this normal map here represents the base normal of my character. So you know, this might have the folds of the cloth and that kind of thing. It's got the really big details. And then these detail normal maps here have the really tight small details. And then this mask defines where each of those are. So if I were doing this on a real character, I would look at my UV layout, and I would paint these various masks in the areas where those materials show up on the character. Okay, like I said, this is the way to not

do it. The reason is I'm sampling uh five different textures here to do the detail. And if I end up with a character that has, let's say, they have eight different types of cloth, then I would have to bring in eight different normal maps and sample nine textures, including my mask, in order to get this effect. The more different types of materials I have, the more texture samples I'm doing, and my shader just starts to get really heavy. If we look at, and this isn't even doing any of the other things that my shader might need to be doing, it's just doing the normal maps. If we take a look at the stats here, you can see that my base pass shader is 186 instructions, and we're doing six texture lookups. So, uh, keep that in mind, 186 and six texture lookups, because we're going to compare that with the next technique that we do, and this is the right way to do it. Alright, so a better way to do this so that we can make this less expensive and allow us to expand.

this and even use more materials without needing to sample a whole bunch more detail. Normal maps is to use something called a texture array, and in order to do this, I'm just going to move all of this stuff out of the way. I'm just going to move all of it up here, and let's start. Let's kind of start fresh because we don't want. We don't want to be using all of that. So, I'm just going to sample another texture here, and in this sample, I'm going to sample something called a texture array. Now, first of all, I need to explain what that is. A texture array is kind of like a stack of textures instead of just having one texture map. I've made an asset where I have multiple texture maps all together in a stack. So, let's go ahead and create one of those. I'm going to open up my content drawer here, switch over to textures, and I'm going to right-click here and pick materials and textures. Then I'm going to come down here, and I'm going to pick texture 2D array.

and it gives me a new array, and I'm just going to call this my cloth array. Then I'm going to open this up, and you can see that immediately it's completely blank. Well, why is that? It's because I haven't added anything to my stack yet. Right here where it says source textures zero array elements, that means my stack is empty. And we know that we're using four different textures. We're using denim, felt, leather, and linen. And so I'm just going to go ahead and click this add button right here and add four slots to put my normal maps in. Then I'm just going to go ahead and add each of those to the slots. So first of all, I have denim, and then felt, leather, and then my linen. Okay, and now I have a stack of four textures, and my preview here shows that there are these different slices that are arranged vertically. And this is not really how the array is going to look. It's just how Unreal has chosen to represent it. There are

actually, there are four textures here that are each square. But it's just showing like a little swatch of each of the four so that I can see that there are more than one texture in the array. Now, if I were using this technique in production, I would probably think about all of the different kinds of characters and the types of outfits they were going to have and what materials those outfits might have. And I would build myself one array that had all the different kinds of details for all the different kinds of materials that my characters might have. And I might end up with a stack of something like 16 different materials. In this case, I'm doing an array of 1024 detail normals, but I'd probably use lower resolution. I'd probably use 128 or 256. And I'd make myself a stack of 16 of those. And then I could use this array for all of the characters in my game. That's how I would go about it. And so I'm just showing you an example of how this can be done, but I hope that

you can take what you learned from this example and apply it to the characters in your game. Now, I might make one array for the characters, and then another array for like the props in the environment, and another array for, uh, I'd split things up into different categories. Okay, so now that I have my array, I need to switch back to my material. Just go ahead and save this here, and I've got my texture sample, and I'm gonna drop this down, and I'm gonna type in my cloth array. All right, so now I'm texturing, or now I'm sampling that array, so we need to take a look at it. Looks like it's doing nothing so far. Let's go ahead and hook this up to our normal socket and see what happens, and you can see I get an error here. It says texture sample uvw input required for texture array sample. So, we need some uvs. So, I'll go ahead and add a texture coordinate node and plug that in, and I'm still getting the same error. It says, uh, texture sample ud uvw input required for.

texture array sample well what is we know that this node is giving us uv but what is a w well a w is the value that it uses to tell which of the items in my array stack i want to sample so what i need to do is add an append here so and i can append a value so i'm going to hold down the one key and click which will give me just a float value and here i've got a value of zero so i'm going to append this so now i've got my uvs coming in here and then this becomes my w value so now if i pass this into my uv socket ah now i get something showing up so i can see my denim normal map all right okay if i change this value to a one now i can see my felt normal map and if i change this value to a two now i can see my leather and if i change it to a three now i can see my linen so by changing this value i'm able to go ahead and

look at all of the different textures that i have in my array. Before i go further, I want to switch over to Unity and take a look at how to set up an array texture in that game engine. Alright, so here we are in Unity, and you can see that I've done things a little bit differently here. I took my four normal maps and combined them into a single texture in Photoshop by just laying them each out in a grid like this. This is sometimes called an atlas texture. So here I have my denim, my felt, my leather, and my linen, and I've combined them all into one texture. Now the next thing that I need to do is make them into an array. So I'm going to come up here to the top of the properties, and first of all where it says texture type, I'm going to pick normal map because that's what this is. And then texture shape, I'm going to pick 2D array. And then it gives me these values for columns and rows, and what it wants to do now is how many

textures do i have in the vertical dimension, and that's two. And then how many textures do I have in the horizontal dimension, and that's also two. So with these settings set, if I come down to the bottom here and hit apply, now you can see that I've turned it into an array. And it says cloth array down here. And then it's using... And then you can see that it says 1024x1024 with four slices. So I've turned this into an array texture. And just like I did in Unreal, I can now use this texture to sample the four different maps in different locations. But in Unity, I had to combine those textures together into one atlas texture in order to turn it into an array. I'll come back to Unity in a second and show you how to create the shader there. But let's switch back to Unreal for now and finish making our array material. Alright, so here we are back in Unreal. And what you can see is that I'm able to sample a single normal map and choose which

normal map. I want to sample with my w value here, but what I really want to do is to be able to tell it, you know, I maybe what I want denim over here, and I want felt over here, and leather over here. I want to be able to define what parts of my character have the different materials. How do I do that? Well, I need to create a texture in Photoshop, a map or a mask that defines where on my character the different materials are. All right, so here we are back in Photoshop, and I have something that looks similar to that color mask that we had from before, but this one is only black and white. Let me explain what's going on here. This dark area here is for one material. Each of these different shades or values is for where I want the different materials to show up. And you might be asking, how did you decide which colors to use for these four different values? Well, let's take a look at this gradient here. I know that I have four normal maps that I want to be able to

blend between and so I want to have these values that represent my four different maps. So what I did is I divided the gradient from black to white into four different quadrants: the darkest ones, the medium two, and then the brightest areas. And what I did is I picked a value that's kind of right in the middle of each of these quadrants. So here my value would be around 31 or 32. Here my value would be right around 96. Here my value would be like 160 something. And then here my value would be like 220 ish. And so I can sample these values and paint them into my mask. And then I can use this mask to determine which of the four normal maps to apply as detailed texture. Now if I were to use something, if I were to use an array that had more elements, I would need to split up my gradient into more than four values. Right now I've got it split into four, but if I were to have 16, then I would need to split it up into 16 and find the

the values along the gradient that go along with that. So basically, in my mask here, I would be painting 16 different hue values or brightness shades instead of four like I'm doing now. Okay, let's switch back to Unreal. Okay, so we're back in Unreal, and I created a new texture sample here. And for this one, I need to find my cloth mask. So here it is. You can see that I've got my four different shades here. And if I just plug this into the base color, you can see that I've got my four different shades of gray that determine where the different values go. But what I want to do is use this to replace this one value here. I know that if I pass something in the range of between 0 and 3, that I will get my different normal maps. But this texture here is in the range of zero to one. And so the first thing that I have to do is multiply this by a value of three so that it can go between 0 and 3. So I'm going to set my mask multiplied

by 3. And now the next thing that I would want to do is I want to round it because I want each of these shades to represent a whole number. So I'm going to add a round. So this first shade is going to get rounded down to zero. The next shade is going to get rounded to one. The third shade is going to get rounded to two. And then the fourth shade is going to get rounded to three. So each of these values, instead of being between zero and one, now they're going to be zero, one, two, and three. So if I pass this value into my w coordinate, now you can see that I've been able to use my mask here to define the areas of my character where the leather should appear, where the felt should appear, where the denim should appear, and where the linen should appear. That's pretty cool. All right. And then for the last little touch, I just need to combine it with my base normal map. So here's the normal map I've been using to represent the base normal map of my character.

i can wire this into base of my blend angle. Correct normals, and then wire this into the additional normal, and then finally connect that to my normal. And now you can see I have a character. Well, I've been calling this a character all along. It's really just the preview cube, um, but this is supposed to represent the character that I have with multiple materials. But you can see that I have my base normals, and then they're also combined with my detail normal, detail normals, and, uh, what I'm also going to do here just really quickly is multiply this by, let's say, by a value of 4, just so that I get a little bit tighter detail so that I can tile my normals. Now you can see I have the leather in this portion, I have the linen in this portion, I have the felt over here, and then up here I have the denim. I think I might have mixed that up a little bit. The denim is over in this part, but anyway, you can see how instead of sampling a whole bunch of

different textures. I was able to combine those textures into a single array texture and make the effect a bunch cheaper. Let's take a look at our instruction count. Now you can see that I'm only using 162 instructions and only doing three texture lookups, as opposed to what I was doing before where I was doing, uh, well, let's connect it up and see. Now I'm doing 186 instructions and six texture lookups. And that's only with four different detail materials. Like I said before, you can go pretty high with these if you want to. You can use, I think I've used up to 16 in the past, and you would not be able to do that with this technique because that's just too many texture samples. But here where you're able to combine them into a single array, you can get an effect that's a lot cheaper. 162 compared with 186 and just three texture samples instead of six. So great technique. All right, let's switch over to unity and I'll show you how to finish up.

there, okay, so here we are in Unity. And you can see that I've created a very similar type of shader. I have the base texture that represents the base normal map of my character, and I have my cloth mask that represents the mask. And in this case, it's looking red. And that's because I have it saved to a texture compression format that just compresses it into a single channel. In this case, it's compressing it into the red channel. So you can see here I'm sampling from the red channel, and then that mask I'm multiplying by 3 so that I can have my range between 0 and 3. And then I round those values so that I can get 0, 1, 2, or 3 out of my mask. And then I use that 0, 1, 2, or 3 value from my mask to tell my 2D array which slice of that stack of normal maps I want to sample. And then for my UV coordinates, I'm just bringing those in, isolating the x and y with my swizzle.

node, and then multiplying these by two. In this case, this is how many times I want to tile my detail normal. And then I'm sampling my array of normal maps here, my cloth array, and I'm using this node to unpack the normal map. And then I'm using the normal blend node to combine that with my base normal, and then passing that into the normal port on my master stack. Let's take a look at our preview here. If I zoom in, you can see that I've got my leather in this portion, my felt here, my linen, and my denim there. So just like I did in Unreal, I'm using this array texture to only make one texture sample and still be able to get all of these different materials. So one more time, just to review, if I were using this in production, I would create myself a set of normal maps and make a stack or an array of those normal maps that represent all of the different kinds of cloth, rubber, metal, and fabric that my characters are going to be made of.

and then I would make an array, and I would sample that array in all of my different character textures. And for each character, I would paint a mask that defines the different materials, what part of the character is made up of each of those different materials. And then I can use that array to make each of those material types look more detailed up close. Now, one important thing about making this array is all of the normal maps need to be the same resolution, and they also need to be the same texture format. So I can't do one that's like a DXT1 compression, and another one that's like a BC7 compression. They all need to match, so their sizes need to match, and their compression types need to match. But once I have that array, then I can sample that in all my different characters, and all I need to do is paint a different mask for each of the characters that determines where those materials are on the characters. And I can get nice tight detail when I get up close, and as I back out, I can still have

decent material with the base texture maps, and you may even find that you can get away with a lower resolution base normal map because you've got extra detail in your detail normal maps. Anyway, I hope you enjoyed today's video and that you learned something, that you learned how to use the sample texture 2D array node in Unity, and that you learned how to create texture arrays in Unreal as well, and that you can now use this technique to make your characters or your props more realistic looking when you get up close. Thanks for watching, everybody. Next week, we're going to come back and do the smooth step video that I've been talking about for a while. I know I got a little bit sidetracked on this normal map tangent, but yeah, be sure to come back next week and we'll talk about the smooth step node. Until then, have a great week and we'll see you next week, everybody. [Music]

Advanced Detail Mapping - Shader Graph Basics - Episode 14

Shader creation and other game dev tips and tricks in Unity and Unreal. My opinions are my own and do not represent the positions of my employer.

Ben Cloward has been a game dev for over 20 years and has been creating shaders since 2003.  He was a co-creator of ShaderFX, a plugin (originally for 3ds Max) that allows artists to create real-time shaders using a node-based interface.  ShaderFX is now built in to Max and Maya as the main way that real-time shaders are created.  Ben was the original author of the Uber shader used in Maya's viewport. In 2007, Ben joined Bioware in Austin, Texas as a Cg Supervisor focused on graphics, performance, and shaders - on titles such as Star Wars The Old Republic, and Anthem.  Ben now works for Unity on the Shader Graph team.  Ben has given numerous talks at industry events such as Siggraph, and GDC.  He loves to share what he has learned.

Ben Cloward

Advanced Detail Mapping: Adding Variety to Material Sets

Detail mapping allows for adding extra surface detail to textures, even when the main textures lose resolution.

The challenge arises when dealing with multiple materials, as the traditional technique only allows for one detail map.

The solution involves creating and blending different detail normal maps for each material, and using a mask texture to apply them to the base textures in a way that allows for a variety of detail depending on the material.

Inefficient Use of Multiple Textures in 3D Material Creation

Using multiple normal maps and texture samples for each detail can make the shader heavy and inefficient.

The current method requires multiple texture lookups and has a high instruction count, which can be burdensome for the shader.

A more cost-effective approach is needed to improve efficiency and scalability in 3D material creation.

Creating a Texture Array for 3D Materials

Using a texture array allows for the stacking of multiple texture maps, such as denim, felt, leather, and linen, providing a more efficient way to handle materials in 3D assets.

The array can be customized to include a variety of material details for different character outfits, potentially containing a stack of 16 different materials for use across all game characters.

When used in production, lower resolutions, like 128 or 256, are recommended for the texture array to optimize performance.

Using Array Textures for Character Design

Arrays can be used for both characters and environment props in game design.

When using a texture array, a texture coordinate node and a float value for the "w" component need to be appended to the UV coordinates to select and display different textures.

In Unity, multiple normal maps can be combined into a single texture and turned into an array by selecting the texture type as a normal map and the texture shape as 2D array in the properties.

Creating an Array Texture for Material Mapping

The process involves creating a 1024x1024 array texture with four slices to map different materials.

In Unity, the textures need to be combined into an atlas texture to form an array, while in Unreal, a Photoshop mask is used to define the material distribution on the character.

The mask consists of different shades of black and white to represent the areas for different materials, with the values chosen based on the gradient from black to white, allowing for the blending between the four different maps.

Adjusting Texture Shades for Character Design

The process involves adjusting the texture shades to create different hue and brightness levels for the character design.

By multiplying the mask values by 3 and rounding them, the range is transformed from 0-1 to 0-3, allowing for more variations.

The final step includes combining the base normal map with the adjusted texture to create a character with multiple material appearances and enhanced detail.

Optimizing Texture Usage for Cost-Efficient Effects

By combining different textures into a single array texture, the instructions are reduced to 162 from 186, and the texture lookups are decreased from six to three.

This technique allows for a more cost-effective effect, using fewer texture samples.

In Unity, the same method is applied, using a similar type of shader to create a stack of normal maps for different materials, ultimately optimizing texture usage for production.

Creating Texture Arrays for Detailed Character Textures

The process involves creating an array to sample different character textures and paint masks for each material type.

It's important for all normal maps to have the same resolution and texture format for the array.

This technique allows for tight detail up close and can make characters look more realistic.