Indoor Environment Creation: Procedural Tools for Game Development

Aardman's Jon and Arkadiusz discuss indoor environment creation in PCG, geometric scripting, and scriptable tools for game development at Unreal Fest.

### Aardman's Approach to Indoor Environment Creation in UE4: PCG, Geometry Script, and Scriptable Tools
* 🎮 Aardman's games team is working on a new **isometric stealth action puzzler** following "Chicken Run: Dawn of the Nugget."
* 🏗 The project heavily uses Unreal Engine 5 frameworks like the **Gameplay Ability System**, with a focus on building tools for **indoor environments**.
* 🛠 The team uses Unreal Engine 5's **Procedural Content Generation (PCG)**, **Geometry Script**, and **Scriptable Tools** to **speed up level building** and **encourage iteration in art and design**.
* 🧱 PCG is used to generate wall meshes based on splines, with **custom logic for mirroring, adding clutter, and placing bricks**.
* 📏 A custom spline sampler is used to discretize splines for wall tessellation, and a dynamic mesh actor is used to **handle curved walls**.
* 🚪 Custom passages are created by inserting **spline control points**, allowing manual control of **door and window placement**.
* The team built a custom spline component with metadata support for various uses, including **controlling corner details** and **wall offsets**.
* 🔄 Initially, the team tried handling curved walls in materials, but ultimately used **Geometry Script for better performance and flexibility**.

### Procedural Content Generation and Tool Development in Unreal Engine
* 🏗 The team developed a procedural content generation (PCG) system in Unreal Engine for creating **complex level geometry**, including **curved walls** and **non-rectilinear rooms**.
* 🛠 They used Geometry Script for **mesh manipulation, dynamic mesh generation, and spline deformation** to create **curved walls that match the level's geometry**.
* 🔧 Scriptable Tools were utilized to create custom editor tools for **quickly generating floors, placing doors, and drawing walls**, improving the level design workflow.
* 📊 The PCG system leveraged data tables for **storing mesh samples** and attribute partitioning for **managing clutter generation in rooms**.
* 🏭 Future work includes **implementing roof generation**, adding **more sampling patterns for wall clutter**, and **improving the floor builder tool**.
* Key takeaways include the **importance of sampling strategies**, using **subgraphs for complex logic**, and the **flexibility of Geometry Script for mesh manipulation**.
* 👨 The team emphasized the **utility of dynamic meshes** for both mobile and non-mobile projects, as well as the **efficiency of Scriptable Tools** for building custom editor tools.
* 📈 The project involved **continuous development and iteration** over the course of a year, with improvements made to both the **PCG system** and the **supporting toolset**.

Right, welcome to Unreal Fest, I guess, for everyone in here. I'm Jon. I'm Arkadiusz. We make games at Aardman, which surprise, surprise, we actually make games. And we're going to talk to you today about indoor environment creation in PCG, geometry script, and a few other other tools. You going to click for me? Yes. So, Ark and I first work together at another studio. We built a fairly comprehensive procedural pipeline to build huge outdoor environments in Houdini and UE4, and it was quite natural for us to pick up this project together to work on more PCG stuff and tooling. We are not a tools team. We're quite a small studio, so this is really like a few sprints' work between us, but it was a lot of fun, and yeah. So Aardman is-- hopefully some of you know, Aardman. It is a historic Bristol-based, in the UK.

animation studio, well known for Shaun the Sheep, Chicken Run, Wallace and Gromit, that sort of thing. And yeah, as I mentioned, we have a games team now, which we founded a few years back, and we also have an interactive XR team. Our ambitions as a studio are to bring some of-- I think our IP is very cultural touchstones, and we're trying to modernize some of the IP for new audiences and build new IP. The studio previously built 11-11 Memories Retold a few years ago, which is quite an indie-style title. And we restructured the studio for new projects, and we started this project in 2022 with Netflix. Scaled up the team to manage that, and that sort of leads into this. Our current project is isometric-like stealth action puzzler. Follows straight on from Aardman's recent film, Chicken Run, Dawn of the Nugget, which hopefully some of you have seen. It is mobile first, PC, console coming later.

But it's been designed from the ground up, well, to manage both, if you believe that's possible. That's a different debate. Yep. We started on 5.1, so without PCG. And currently on 5.3, which we are upgrading soon. But we have got a major milestone at the moment, so that's on hold. The project heavily uses the gameplay ability system, many other UE5 frameworks. So that would be lots to talk about. But on this talk, we're going to focus on actually building invite-like tools to build our indoor environments. And this is interesting. Ark can talk about it in a sec because a lot of the ideal example situations of PCG are not that sort of environment. We have short development cycles for this project, short development cycle. We did not understand gameplay when we started building tools, which obviously, is difficult. And do we understand gameplay yet? I think so. Yeah. The game just was not fun at the beginning.

But yeah, we didn't understand gameplay, so we wanted to build tools that were really flexible that we could iterate really fast on. And we would kill our darlings, so to speak. So we could just throw away as much content as we wanted and iterate quickly. So we've got a little short. This is really early footage. [VIDEO PLAYBACK] [MUSIC PLAYING] [END PLAYBACK] Cool, thanks. Thanks a lot. [APPLAUSE] So obviously, that says PC and console. Pretend that doesn't say that for this. We're not video editors, so that wasn't possible. So we are not trying to suggest any specific algorithm-- did I say that right? Practiced this so many times. --or approaches. This is more of a quick review of how we've rapidly built tools for our team with these frameworks. So we have PCG. It is almost an exponential learning curve-- a lot of depth, highly flexible, scriptable, procedural content.

framework. I actually had to read that. It is data-driven and getting more so. Then we have geometric script. That is a geometric processing framework -- fast and extensible implementation for many general mesh operations and manipulation. So if any of you used modeling mode before, it's something similar. I mean, it's basically an automated version of that. It's a pretty good way to find out what features you can do in there. Yeah, and it's really good for baking. Last thing we're going to talk about scriptable tools. So it's like a simple tooling to build quick editor mode tools. And basically, encapsulated the life cycle of our tools. So it was just very easy user interface for our designers and artists to just have everything in one place to play around. So I have -- we have -- Sort of just saying artist and designers just play around. Is that what you're saying? Yeah, yeah, yeah. So we have a couple of crowd questions.

Any of you used PCG already? Oh, wow. That's actually quite a lot. And if you use the geometry script? That's a bit less, but yes, still good. And any of you tried scriptable tools? OK, a couple of people. We have some people. That's cool. Yeah, that's good. So obviously, there are many different ways to build tools in Unreal Engine 5. Our requirements were that we speed up level building, encourage iteration and art and design. They really wanted to have production-ready assets during white boxing, and we know how all that goes. But actually, these tools-- both, I mean, PCG geometry script and scriptable edit tools, they seem to actually fit the bill. It was very nice to actually upgrade stuff on the fly, and people would see, actually, these white boxing using those production-ready assets. So all of this fit the bill. What we didn't want to use were third-party stuff.

It's also for mobile performance, as well. Yeah, yeah. For mobile performance, as well, yeah. They actually-- they are really good for that. We didn't want to use third-party stuff because it was-- like Houdini because it's just hard to maintain. And then if you go into C++ or blueprints to build from scratch, it's just a lot of boilerplate. In summary, it was just really hard-- it is really hard to convince people to try new tools. So when PCG was introduced for the first time, there was this one-hour presentation, one-hour video by Epic, and there was like this five-second clip of this building. And we were like, yeah, we want this, although it was just five seconds. It was a nice validation, actually. We talked about using it, but we thought, would this map well to what we were trying to do? Or were we just trying to fit a round peg into a square hole?

And yeah, this validated it a little bit before we dug into it. Yeah. I mean, and this is, as well, fun for us because we both work on procedural gen together. And Jon's academic background is in these areas, as well. And so, our whole approach to building rooms is based around splines. Our actor blueprints, they contain its own spline and PCG component, in which the PCG component has biomes, different styles for each side; some niche features like, let's say, mirroring; some custom clutter on walls and bricks to break tiling; and we will go, as well, through custom spline work we did. Lastly, passages. Because although we make rooms, we want to connect them somehow. And keep in mind, as already Jon mentioned, we didn't upgrade to 5.4 yet. So all of this you see is on 5.3. Here we are going to give you an overview of our room builder tooling.

What is really important is that we are using data tables. As we were expanding the PCG graph, we didn't want artists to mangle too much. So we decided initially to have a data table, which stores all the biomes and styles, and we actually really did like it. So for artists to just jump in, fill a style, add extra details to what they wanted to use, it was just amazing. And then we have a blueprint that covers corners, curved meshes using geometry script, which we'll get back to as well, and our PCG graph component. And although this looks maybe too simple, but I'll actually show you how it looks in editor. So this is our wall PCG graph. What we do, because our blueprint actor has one spline, so we get that spline. We use our custom spline sampler, which Jon will get back to you later. And then we just filter out which segments of the spine are linear or curved. The reason is we just want to process the linear ones with PCG.

and the curved ones with geometry script. After that, we do a little bit of filtering. So we either put our passages or some custom meshes, even some hero assets, if I'm correct. Then after this, once we finish, what we are left with are just where the points for the walls are. So yeah, you specifically filtered out passages, as well, by this. Yeah, yeah, yeah, passages. Yes, I did say that, yes. So this is actually the main core logic of this blueprint where we take our points, we give them an attribute of which mesh it should use, which material it should use. And then in the mesh spawner, we check these attributes, and it will just use a template to just spawn them. But the reason why we have it twice, it's because we get the same points. But we want it to build on both sides different styles to have control over that. So as we separate this out, we just make a copy of these points.

and we just rotate them by 180 degrees. That's literally it. From this point, we started adding more custom features. So in this case, we have mirroring. And what we do in the data table, we have an attribute to say, do we want to mirror this specific style? And if we say yes, what we decided to do is we would just literally scratching the surface of PCG blueprint elements. So they felt very overwhelming and complex. But in all honesty, once you understand that there is a function called point loop body, which basically loops over each point, what we discovered is that if you break the endpoint variable, you can get something called metadata entry. And what this is, this is actually the index you see in PCG of every point. So what we did, we took a modular. So we just filter every second. And we have a select node to just say either we don't flip it, or flip it. Put it back to the PCG point and just return it.

After this, we have a couple, as well, extra just-added features which was requested [INAUDIBLE]. So for the input, for our crown logic, we take our wall points and whatever we filtered out previously for passages and custom meshes, and we first check with our data table. Does this specific style use crowns? Then we add the mesh. And then, because we didn't want this specific thing to be double-sided, we actually choose from which style we want the crowns to be spawned. After that, we just transform them a little bit higher on top of the walls. And again, same logic. Use the string to just spawn the mesh. Next section is actually spawning clutter on walls. So again, we just check. Do you want it for this side or the other side? We then give them some custom offsets if they require. Then if they were mirrored, we actually unmirror them. And after this, we have this nice spacing logic. So we can either go with select points node, which is built-in,

which will randomly choose a selection of points. You can, as well, give your attributes from your data sheet to be referenced so you can control it. Or you can do another custom blueprint. So this one is, as well, doing the same thing. We are getting our points. We are getting our index. And instead of filtering every second, like for mirroring, we are making a new attribute which you can easily build in. And then we are just defining-- we are selecting, is this true or not? And what this does, if it's true, the point will be kept. If it's false, the point will be discarded. So after that, as well, our data table can control for the certain style how many walls we want. And the next section is our specific clutter. So in this case, artists really wanted to just have clutters on specific style. So we are checking-- Wall clutter. Wall clutter, yes. So our logic basically checks if the point is

supposed to use this specific mesh. If it's using this specific mesh, spawn this type of clutter. If it's this specific mesh, spawn this type of clutter, and so on. And last and least, we have bricks. That's not the saying. I know. So we have bricks. And for bricks, there is a node called input data table, which is different to the new load data table. But what this does, you can actually import a specific set of points. So we made a data table with hand-placed points of typical brickwork. You say we. I definitely did not do that. Yeah, you did. Shh. And then what we actually do, we take our wall points, and we copied those for each wall point. After that, we again do a random selection. Could be defined by data table. Could be defined even per blueprint. You can override this. And then we do some scaling. After that, there is this interesting system for static mesh spawners

where you can actually make a select type -- a PCG mesh select weighted by category. And what this does is that if you have a certain category that it expects, it will look through the array and see which meshes should be spawned for this type of array. So in our case, we have a manor garden style, and then it will just spawn these four meshes. If we would have a different style, it would look for a different index. And that's it for this wall. So here, we have examples of what we did. So we have the two-sided style. In the first image, you can see it, so we can independently control which style we want on the wall. After that, we have our custom overrides for any kind of custom mesh you want. It just checks the wall and checks what should be spawned instead. Next is mirroring. And yeah, this is a specific style where we just did one side of the wall, and the next one is just flipped so it looks nicer.

After that is clutter. As well, could be manually overridden if an artist is unhappy. They usually were unhappy with how it looked. After that is crowns. So as well, again, we just have these extra meshes on top to add a little bit of style. And last and least are bricks. So as you can see, this is like initial pattern. And then we filter out, and we can scale them. And yeah, the brick styling, which is pretty cool, in my opinion. I'll briefly talk about the spline sampler that Ark mentioned. So what Ark said at the beginning is, our room building tool is essentially based around splines. But really, it's actually based around the discretization of splines. So we are generating PCG samples along the spline, and we are then tessellating meshes along that spline to make it look like a contiguous sort of room or wall. And there weren't really any samplers that fit the bill for us easily out of the box, so we built our own.

This is actually very simple. You extend UPCG settings, which is a class in the PCG framework and plugin. And you can set up custom params. That's quite straightforward. And essentially, we are overriding the F subdivision step sampler, which is-- yeah, a class which extends F step sampler. Within the step function, that's where most of our logic is for doing this. So essentially, the normal input for a subdivision sampler is to take N points and sample that uniformly according to some rules along the spline. But we wanted to do it according to a mesh. So whatever mesh we would feed into it, or set of meshes, we would get an output. So basically, instead of computing the subdivisions-- sorry, instead of passing the subdivisions per segment as a parameter, we compute that inside this function. And then we just compute the out transform, including the distance along the spline and the scale based on the mesh that we input. And then fixing up indexing, which is a little bit

different to the normal subdivision sampler. So that gives us a really simple way to just automatically tessellate any spline with the mesh that we put in. What we also can do there, we have a slightly more accurate curvature computation. We can compute some other stuff in the step sampler, as well, which is quite useful. These were all written as attributes. As Ark mentioned, PCG is similar to a pipeline where you're reading and writing from attributes all the way through that pipeline to allow you to do filtering and different stuff along that PCG pipeline. So having our custom sampler means we can compute new attributes alongside that. And you can see the gif here. By the way, this talk is basically just gifs. So if you like gifs, then that's great, as you've already noticed. If you don't, then this-- Or jifs. --not the talk for you. Yeah, yeah. Cool. Oh, OK. So yeah, we needed passages for our rooms

because we just need to connect them somehow. So these were our doors, double doors, vents, and even artists have-- yeah, it just-- we add windows to it so we can manually control where they are. We're like, yeah, yeah, OK. The simple solution was just replace the PCG wall sample with a passage. However, this restricted design because they just couldn't tweak a little bit the doors to the left, to the right. Also, it's super fiddly. Yeah, it is super fiddly. So instead, we just said, yeah, just place the passage way you want. And what we did is we insert a spline control point either side of the passage. And because of how our spline sample works, it will auto compensate for the measures around it, as you can see in the second gif. Like right at the end of the second gif, which you have to wait for the entire time. That was really well done. Yeah.

There we go. So that's the different approaches, that gif. Just to let you know, that's not the approach of fitting a passage. Like you just click and it just fits the passage. So that's the work in progress. We thought we'd briefly talk about the fact we use custom splines, which I'm sure most projects do. So we built our custom spline component, and that obviously has spline metadata, and spline metadata details. The custom spline we used for various different things throughout the project, in this case, we just embed-- or most of the use cases for this we do some interpolation, but most of it is embedding metadata in our spline control points. As probably a lot of you know, there's a lot of boilerplate to implement this in Unreal, which is a bit fiddly, but it's quite straightforward. This allows us to have custom corners per control point. So these might be like turrets, or they might just be tiny little things.

like in this gif here. And yeah, this is great because it breaks up the thing, and designers can control whether this appears on every corner or not. And sort of manually control the offset. It's also sided. So as you move the wall around, it generates the offset on the right side of the wall. We also have, again, spline component visualizer, which we use for various functions throughout the project. This allows, as I'm sure, again, everyone knows, this allows us to decouple functionality from the spline component or add easily functionality to the spline component. An example of this is we-- because we've got these large rooms and large walls, and obviously, people don't use the tools as we intended. So instead of building small little cute rooms and stuff, and then having that, they build enormous walls, like hundreds and hundreds of meters long, and then complain that the pivot's too far away, et cetera, et cetera.

So we just have a tool in the spline component visualizer that allows us to split a spline, split a wall. And it just generates two PCG graphs, regenerates themselves, inserts control points, and changes the pivots, and stuff. And just as a right-click menu. So that's super good for functionality there. So quickly, how we handle curved walls. So Ark mentioned curved walls in the PCG graph. Our original idea to solve this seemed like a super cool idea. It was a bad idea. Yeah, it was. We were convinced that this was the right idea, as many of us are on projects. And the idea, I think it sounds sensible, but the idea was basically to handle it all in PCG and materials. So we would just treat a wall segment as a curved wall segment. We would sample it in the PCG graph as before, but then in material, curve that mesh locally, that instance locally. Did you mention-- so all these are obviously instanced static meshes.

And we would curve the mesh locally and in the material to bend it to the spline, essentially, using custom primitive data, which you can send in PCG. So the problem with that was that we would-- well, it was quite complicated to do in the material. It's quite complicated to get it to follow a spline locally. You had to send quite a lot of data. Mass was fairly messy. And also, this has actually turned out to be quite a lot of performance overhead to do that much vertex transformation on the GPU every frame. So our interim solution was to filter out the curve points. So we looked at the attribute sheet, as Ark mentioned, filter out any points that don't have negative or positive curvature, and throw them away, pretty much. And then insert a spline mesh, and that would follow it. We could set up the materials, basically. And that looks fine. Actually, it doesn't look fine because you can't flip elements. We can't have custom materials.

It's problematic. And if the curvature of the mesh is high, we get lots of errors because the spline mesh won't handle that properly. So our solution was to build this in geometry script. So we used the same approach with the spline mesh. So we filter out the points, throw that away. And then we feed it, so Ark then parameterizes this geometry script class we have. So that will still tile this instance mesh as we want across the curve. It'll handle backwards and forwards, and it will curve it properly to the spline. But also, we can do things like subdivision, and decimation, and stuff to make it curve really well. So a single draw call is sufficient. We have the dynamic mesh actor, which I'll talk about in a second, which allows like a non-destructive workflow with the PCG graph. So it handled everything really nicely, and I'll just dig into that a little bit now. So this is an example here of--

this is a PCG wall graph in the gif, and just modifying the curvature of the spline. And this is generating a geometry script curved wall on the fly, which is baked down to a static mesh on the fly, as well, but is fully reversible. A bit like-- I don't know if anyone's played around with Lyra. They do something called cold storage where they maintain the references for the dynamic mesh and the static mesh so you can flip back and forwards when you're moving windows around. So geometry script is based on dynamic meshes, which, as you can tell, you can guess what the difference is there with the static mesh. It is based-- like in blueprint, you will extend a dynamic generator, dynamic mesh actor. And normally, instead of then tying stuff into a construction script, you'd use the Unreal editor on rebuild generated mesh function, which just performantly handles that rebuild at runtime in the editor. There's some other functions in this class to handle copying back and forth between static mesh actors

and showing progress in editor if it's a large rebuild. But broadly speaking, this is the sort of environment you'll be working in. And I'm just going to briefly talk-- so the core approach to this is that we sample the curve segment of the spline, spawn static meshes, and then deform them locally to match the spline. Then we bake this down and release all our assets. And I think-- here we go. So I'm just going to show you this blueprint, as well, just very quickly. So essentially, this is our function, the rebuild generated meshes. We do a bunch of allocation of our dynamic meshes here. And then first of all, we handle the front side of the spline to handle our inside meshes or outside meshes, depending. So as you can see, the act is quite simple. We have a front spline and back spline. Then most of the functionality is in this build and aggregate mesh function here. Inside this, we essentially copy from the static mesh

into our dynamic mesh object. Then we apply uniform tessellation here. So this allows us to, obviously, really increase the density of the mesh, which allows us to then deform the mesh accurately, and then simplify it afterwards. These are just standard geometry script functions. So then this is a custom function here. So the mesh sample spline into transforms. So there are some spline sampling functions here, but again, we wrote our own for this. This allows us to sample the spline according to the mesh size in whatever access we specify. And from that, we get out these frame times along the spline, and a uniform scale for the spline, as well. This also allows us to adjust for where the pivot is on the input mesh depending on how the artist has decided to build it. And essentially, then, we copy this template mesh, which we've subdivided once. So we only subdivide once in every process. We copy this to our spawning mesh, what we're going to modify.

This is a bit of a hack here just to handle the mirroring Ark showed on the PCG graph like where inside castle walls are flipped every other thing. Hack, yes. Yeah. Which was a hack in that, as well. And this is where we-- oh, sorry, this is the interesting function here where we deform the mesh to follow the spline. I'm not going to go into it here. It's a bit too much to go into, but essentially, it's some fairly trivial linear algebra to follow the spline to deform the mesh locally, scale it in X, as well, which is our axis for our wall pieces, and to deform it locally to follow the spline. Pretty much similar to the function used in spline mesh component. Then we append it to our aggregated target mesh, which is another dynamic mesh. We set up materials, and UVing, and recompute our normals here. We do this once afterwards, so we get smooth normals, obviously, across the curved segment.

And then, so basically, we do that for the front side of the mesh. We then have a custom function to reverse the spline, do it for the back side of the mesh with a different source static mesh. Then we aggregate these two meshes. We apply some planar simplification. The reason why we use the planar simplification here-- there's a few simplification methods-- is this allows us to give an angle for control, which means we can almost enforce the quality of how we're using the curvature. So it allows us to have a minimum quality there for that, which is useful. Then we release all our resources. We have this function here. We don't use it here. Like Ark actually calls this from the PCG actor, which also owns the graph, but this handles the baking to a static mesh-- so the auto naming, adding to source control, et cetera, et cetera, computing collision, stuff like that. So that's then called, and the PCG graph actor, that manages the state.

like cold storage back and forth between the dynamic meshes stored here and the static meshes output from this process. So that is a quick run-through of what we're doing in the curved mesh stuff. And here is just a quick thing. This was a gif, but it was so hard to follow I couldn't describe it. So I actually converted it to images, which is a bit of a breath of fresh air. So you can see on the left-- I'll go through this pretty quickly. On the left, we have the normal mesh if we don't do any subdivision and decimation afterwards. And you can see, it's an optimized mesh designed for mobile. These are all instances, by the way. Well, no, they're not anymore because it's baked down. So this is a normal wireframe. You can see subdivided we have this very high poly count, but we match the curvature. You can see the errors on the leftmost curved wall. Like if we don't subdivide first, it looks rubbish.

And then we can decimate afterwards. So these functions here are all part of geometry script, right? Like this is not our novelty, but it's just really useful to be able to do this in geometry script really easily, and it's much more efficient and useful to us than using the spline mesh. Quick other algorithm we built. So we built something to enable-- like we have lots of non-rectilinear rooms, or all sorts of odd shapes. We have curved rooms. And because you're looking top-down, you can see the extent of all the floors. So we built this quick tool to allow you to generate floors in the level very quickly. This is a custom algorithm, so I won't go through it in much detail because it's less relevant to this talk. But it does use geometry script, so we thought it'd be interesting to mention. This allows really quick automatic generation of floors. You basically click, it uses a geometric floodfill. It builds a data structure that allows fast neighborhood lookups.

so you can do more interesting stuff with that data. But essentially, we build a boundary for this, generate a poly from the boundary. Most of the complexity in this algorithm is handling edge cases. As you can imagine, with really concave polys and stuff, it's pretty nasty, especially when we're light tracing out into the wall geometry to figure out where the floor is. But basically, this is not super clever. It just allows artists and designers to quickly build floors for a level. So this is all built around geometry script, as well. And we generate a dynamic mesh from this poly, which again, we can use this-- it's almost a cold storage method to go back and forwards between the static mesh and our dynamic mesh. Except actually, in this case, we throw the dynamic mesh away to be a bit more optimal. If you sample the floor, as you can imagine, there is a grid, so it's not completely deterministic. But if you sample the point at the same place.

it's completely deterministic when we regenerate this floor. So that's all we store when we go back and forth between this. So you can easily move a wall and regenerate this, and it doesn't use the same footprint in your project. This obviously bakes all these static meshes automatically. You can see the naming when you build this, it generates this and writes it to the project structure, and et cetera. It does a uniform remesh on it. I think you can see it in the last thing actually. You can see when it builds this mesh we do a remesh of it, which is pretty optimal for the project. You can see when I just dragged this up. Yeah, gifs, gifs, more gifs, have to wait. There we go. So it produces a fairly simple thing. We could probably make that more simple. We also have this space delimiter thing. So using this neighborhood structure, we can detect gaps. So if someone set a room up wrong, it gives them a giant instruction.

like a massive red arrow to show them where they've screwed up the room, basically. We also generate a boundary spline from this. And the reason that's interesting is we use it for room clutter and other things down the line. The reason why we don't just use-- yeah, the reason why we don't have that boundary spline anyway is because, as we said, we thought people would just build rooms, right? But actually, most of the time they just add a little bit on. They build a wall, and then it creates a room implicitly from the connection to other rooms. So building this boundary spline is really useful for later clutter generation, which we can talk about now. Yeah. OK, so our room clutter is very artist-led because they just wanted to place everything by hand. So we said, yeah, go for it. And as they were building, we've noticed that there were a lot of rooms where they were stacking boxes on top and stuff.

And we were like, you know, you can just use physics simulation for this. And they were happy with it, but then, we've decided to actually go deeper into PCG. So at this point, I've discovered this three dimensionality of attribute sheets. And what is really cool about this, you could create a data table where you have each of your mesh are stored in a different row with some data. And then what you can do, you can load that data table into PCG, and then use this amazing node called attribute partition. And what it will do, it will take all these-- if you sample by the mesh, if you use the variable from the mesh, it will split this attribute sheet, each point, into an individual attribute sheets, each one containing one point. I can see this somehow blowing your mind even talking about it. I mean, I'm just-- it was really exciting. It really did blow my mind. But what is cool about this, you can actually then

go into subgraphs, which are, as well, really, really useful. So there is a node called loop subgraphs. So what this does, this will actually loop independently per attribute sheet you feed to it. So in the gif you see, we are actually having a room. We are filtering some logic. But then once we filter through all of the clutter, we put them all together so they are bunched up overlapping. After that, we use a prune node-- yeah, prune node-- to just make sure they don't overlap. We spawn them as static meshes. Then we run simulation and just take the output. And yeah, artists were quite happy with this. Yeah, we kind of went rogue on this one. Because normally, we're just following whatever-- like reviewing whatever tools are artists and designers want, and then talking to them about it. But this one, we were like, I think this would be useful, and we wanted to try out some new stuff.

So obviously, our art director would probably look at what you've generated here and cry. But it's a good starting point. I think this is also on our previous procedural project that we worked on in our last company. Having a basis for artists and designers to start from is sometimes part of the goal. So generating enough for them that it's not scary to start on this level is often really useful, and I think that fills that. And you can easily build rules for different rooms. So in our case, we could build clutter for storage rooms, build for kitchen, and stuff like that. So now we're going to give a quick talk about scriptable tools, which interestingly, was the least known about thing in this room, I think. It's very simple. It is a framework for building really trivial custom editor modes, basically. And basically, it allows you to-- as a programmer, you can separate concerns, sort of encapsulate stuff differently. So it's almost, you can very trivially

follow some sort of MVC pattern with your tooling. So you can build your tool completely separately, and then build your scriptable tool, and just wrap functionality with it, which makes it really useful to build. It's modal, so it has a pretty well-managed lifecycle, though I will go into that in a bit. It's amazing for building quick stuff. You can see we built a few tools here as an example to show you what's going on. The three main classes that you'll end up using are scriptable single click tool, scriptable click drag tool, and-- I'll have to read this one out-- editor scriptable interface tool interactive tool property set. That's a really long name. Yeah. I think this is proof that Epic don't review class names in their process. So if anyone's ever wondering, I think we've proven that. And yeah, feature-wise, you get life cycle support, which you have to put some effort into; click, drag, hover.

It's largely event-driven, so it's really straightforward to use. You can add gizmos to anything you like, property watchers. So if you change data, you can rebuild automatically. And you might be scratching your head wondering what I'm actually talking about here, but I'll show you the tool in the next slide. Messaging help on screen; rendering-- simple, hard; serialization through your property set; et cetera. Actually, that property set thing you don't have to use, but it's another way to separate your data from your tools. So it's good to use it, I think. So this is an example of using the tool. So you can see the interface here. So if you choose a tool on the left, bit like if you've used modeling-- if you've used any editor mode, really, you choose your tool here and set it up. And you can see on the left, I realize there's three gifs, and there's a lot going on here.

So if you try and follow what I'm talking about. So the left one is the wall builder, which you can see in the scriptable editor tool. So this is our wall builder tool-- room builder, sorry-- wrapped with a scriptable editor tool. So you can see I've got gizmos on here by default, which is probably stupid. It makes it quite confusing to see what I'm doing, but you can just toggle this on and off. And when the gif restarts, I'll talk through it. So you can see I'm just placing points here. Yeah, now placing points here. I can modify them in real time with the gizmos. You see, I label the head and tail with message text to make it clearer. And you can see also I've turned snapping on there. So this is handled through a modifier, through the control modifier. And that allows snapping to tiles, which is super useful. That's not part of scriptable tools, really, the snapping.

but the modifiers are. Then that's a shift modifier to insert a point. It's really flexible for what you're doing. You can also-- and we've done stuff so you can just add to the head or the tail of the spline. It's a super useful interface for drawing splines. Then we've got our passage tool here where-- so Ark mentioned the complexity of passages and how they're a little bit tricky to set up. So this tool just simplifies that. So you just choose the door you want and just click anywhere on the level. It just inserts it in a wall. So it takes 10 seconds to put doors in the entire level, which is super useful. Then the floor tool. Finally, you can see when the gif restarts, that I just click around, generate all these polys for these floors, which this is obviously-- like the floors in this level are quite rectilinear-- in fact, they're quite rectangular, I'd say. So it's not a super good example, but it is dense and complicated.

and there's loads of geometry and stuff. And then I just generate all those, bake it down instantly, and then you can drag and drop materials into the scene and completely build out all the floors in the scene super quickly. And again, part of that utility is from how good scriptable tools are just as a really quick tooling system. This is just our learnings from using scriptable tools, which I thought would be interesting to briefly talk through. So modifiers don't actually work super well out of the box. They work well in terms of when you're setting up your blueprint scriptable function-- which again, is just sort of a bunch of events. Like you have a startup event-- it doesn't handle modifiers too well. So you'll get modifiers like shift key control, Alt, which is what it supports out of the box, but they will only work on like your startup and your click events. So if you're having a mouse hovering, which is probably a lot of the way

you will use the tool that won't update your modifiers. So I thought I'd just put this up here to give an example. So if you extend this tool and just override the setup function, then you can just bind events to some of the other events happening. So you actually get real-time modifiers. Otherwise, if you're hovering, not clicking, and you press shift or control, you won't get these updates. So that's useful. We added support for viewport grid snapping into all of our tools and throughout the project. So this is just this very simple function that grabs that from the viewport. So when the user's just changing the viewport, you can just grab that value. So whatever that value is will be used throughout our scriptable tools as well. And then as you can see in the blueprint thing there, we can just grab that value and snap to grid. State saving is a little bit nuanced. You have to do things in a certain order to make sure that state saving works.

So you can serialize data between sessions in your property set, but there's some nuance there, so that is a bit fiddly. Complex watches don't work. So you can drive property change events, basically, and they don't work for complex types. So you have to either roll your own. So I assume if it's a basic type, you will get an event from that. If it's not a complex type, that probably won't work. Just bear that in mind. Clean up very carefully. So it is modal. It does handle its life cycle, but there's a shutdown event which you will have to manage stuff from. It's very simple. So you can get selected actor, as well, in a scriptable tool. So you can actually load an actor in. You can load your complex wall in and just edit it. So obviously, if you're loading that in, you don't want to roll back to no content, so you need to handle that rollback properly, obviously.

Very simple to wrap stuff for. Not that we found anyway, there are no default delegates for when a PCG graph is rebuilt. So if you're wrapping PCG stuff with this tool, you need to write your own stuff, probably to-- we can find it anyway-- to handle when a graph is being regenerated. So that is, obviously, a process. You can't by default hook into that. So yeah, extra bits. We've built an editor widget to help us handle regeneration because it was a little bit difficult to updating the PCG, then regenerated the map per map. One thing, as well, we did try to fix, there was a small bug when you tried to do control z undo. It would leave ghost meshes in PCG. So as well, we try to fix that. The other thing we've added were bushes, as well. This was very niche for one specific map. Somebody was asking to do kind of maze out of foliage.

So yeah, we just made these guide splines you can use for PCG. And now, as you see in this maze, we will have to change because you know the solution. He's been excited about telling that joke. I'm not sure why. Yeah. Last thing are tunnels. And as well, this is the same idea with having trade lines, handled by PCG, and then curved by geometry script. Cool. So just a couple of slides to wrap up on future work. Do you want to go first? Yeah so for PCG, we would like to have more of a global way of manage, and regenerate, and test the PCG graphs in the project. We wanted to do-- The reason for this, by the way, is if you build your PCG graph, and you generate some stuff on a level, and then you forget that level, don't work on it for three months, and you've changed the graph a lot, that won't be regenerated in that level.

And also, we don't know quite how to handle that. We don't just want to regenerate anything because that's a QA nightmare. To then regenerate everything and not go and test it. So it's a bit nuanced to us how to handle that, I think. Yeah. Another thing we really want to do is roof generation. So currently, we could just use the flow builder, have a flat roof, and just move it up. But we would like to actually try to do some kind of more complex stuff, and as well, build different ones for different styles if it's possible. Yeah. I think this is just out of scope of the project, really. Because if we're building roofs for rectilinear buildings, then it's reasonably straightforward. As soon as you add curvature, it becomes even more tricky. We just didn't have the time to work on this, really, so that would be something we would like to do. Sampling patterns. So Ark mentioned we have this data table with samples for brickwork.

We'd like to use-- there is actually a plug-in called PCG geometry script interop, which allows PCG to call some of the sampling functions in the geometry script library. We'd probably like to add to that to add some more noise patterns and industrial patterns to make wall clutter. We currently don't handle passages on curved walls. This is not because it doesn't work. Obviously, this would actually be quite straightforward carving into the mesh with a Boolean op or whatever in geometry script. We just haven't done it. We haven't had time. And floor builder. The artists want ways to explicitly define split room floors-- wasn't described very well. So if you have some giant room, they want to be able to draw a boundary and split that into two things. So we'll probably do that. Also, not auto-regeneration as such. That's a misnomer, really. But be able to click a button to regenerate it based on what the room changes.

Hierarchical fog of war is something I'm not going to go through because we don't have time, but there we go. Oh yeah, the takeaways. So it is great to build tools around these amazing frameworks. So if you want to go through, consider your sampling strategies, how you want to sample the data, consider using subgraphs. They are very, very powerful, and as well, use data tables and data assets. They really help you to do more versatile work. That's got a lot more flexible in 5.4-- well, 5.3 and 5.4, utility of those. That wasn't available, really, when we started. Geometry script is super flexible. Dynamic mesh is very useful. I think there's real utility in non-mobile projects for using dynamic meshes in release, as well. I think someone told me the other day that you can use that with Lumen now. And yeah, I think there's real utility there.

But also, like all the tools to bake stuff down, it can be very efficient for mobile. And just having the flexibility to do a lot of these mesh operations in editor is super useful. Scriptable tools, as you've seen, utility is incredible for something that's so easy to build. You can build these in like an hour or so, and have a tool that is far more usable than the tool you had before. And yeah, just to say, really, this has been continued development through the project. So the original thing was pretty bare bones. You could see, I think, the life cycle from Ark's PCG graph. As right at the beginning, we were both looking at that to figure it out. And then Ark's been working on that over the year, and I guess his knowledge of that has changed a lot. And also, the tooling for that with 5.3 and 5.4 has changed a lot. But this is really just us building these tools throughout the year.

like just in our spare time, really. Like moments here and there to try and build them out. But yeah. Anyway, thank you very much. And that was the end of our-- Thank you. [APPLAUSE]

Streamlining Indoor Environment Creation with PCG and Geometry Script in UE5 | Unreal Fest 2024

Welcome to the official Unreal Engine channel!

Unreal Engine is a complete suite of real-time 3D tools made by developers, for developers. Create and take your content anywhere, from PC, console, mobile and VR games to cinematic experiences, visualizations and training applications. With Unreal Engine you get everything you need to start, ship, grow and stand out from the crowd. 

Click link below to download Unreal Engine.

Unreal Engine

Aardman's Approach to Indoor Environment Creation with PCG

Aardman, known for animations like Wallace and Gromit, is venturing into game development with a new project in 2022 with Netflix, following their previous title "11-11 Memories Retold.

" The studio's current project is a mobile-first, PC and console later, isometric-like stealth action puzzler, developed on Unreal Engine 5, heavily using the gameplay ability system and other UE5 frameworks.

The studio's team, despite being small, has been focused on rapidly building flexible tools for indoor environment creation, leveraging procedural content generation (PCG), geometric scripting, and scriptable tools within Unreal Engine 5.

These tools were chosen to speed up level building, encourage iteration in art and design, and provide production-ready assets during white boxing, all while ensuring mobile performance and avoiding the use of third-party software.

Overview of Room Building with Procedural Generation

The room building process is based on splines and PCG components, allowing for biomes, different styles, mirroring, and custom clutter.

The use of data tables helps in storing biomes and styles for artists to easily add extra details.

The blueprint covers linear and curved meshes, with a logic for processing walls and adding custom features like mirroring and wall clutter.

The process also includes adding bricks using hand-placed points and a random selection, providing an interesting system for static mesh spawners.

Custom PCG Mesh Selection and Spline Sampling in Room Building

A custom PCG mesh select allows for specific mesh spawning based on categories, providing independent control over wall styles.

The team also developed a custom spline sampler for tessellating meshes along the spline, allowing for a contiguous room or wall appearance.

The project includes a custom spline component for embedding metadata in spline control points, allowing designers to control the appearance and offset of specific features.

Simplifying the Process of Handling Curved Walls in PCG Graphs

A new tool in the spline component visualizer allows for the splitting of walls and splines, generating two PCG graphs.

The original idea of handling curved wall segments in PCG and materials turned out to be complicated and caused performance overhead.

The interim solution involved filtering out curve points and using a geometry script to properly curve the mesh to the spline, allowing for subdivision and decimation.

This approach also enabled a non-destructive workflow with the PCG graph, using dynamic mesh actors to handle the process effectively.

Mesh Deformation and Floor Generation Using Geometry Script

A function is used to deform the mesh to follow a spline, then it is appended to an aggregated target mesh with materials and UVing.

After that, a custom function is used to reverse the spline and deform the back side of the mesh, which is then aggregated with the front side.

The geometry script is also used to quickly generate floors in non-rectilinear rooms, using a custom algorithm that allows for fast floor generation and dynamic mesh creation, optimizing the process of moving walls and regenerating the floor.

The generated static meshes are automatically named and undergo uniform remeshing for optimal use in the project.

Additionally, a space delimiter is used to detect gaps in the room setup.

Room Clutter Generation and Scriptable Tools

Room clutter generation using PCG and attribute sheets allows for efficient and realistic placement of objects in 3D space.

The use of scriptable tools provides a framework for building custom editor modes, separating concerns, and encapsulating functionality.

Scriptable tools offer lifecycle support, event-driven behavior, and the ability to add gizmos and property watchers for automatic rebuilding.

The tool allows real-time modification of splines and offers a simplified process for inserting doors into walls.

It also enables quick generation and building of floors, with the ability to easily add materials to the scene.

The tool's setup function may need to be overridden to ensure real-time modifiers work properly, and support for viewport grid snapping has been added.

State saving and complex watches have some nuances that require careful handling.

The tool allows for loading and editing of complex walls, but it lacks default delegates for when a PCG graph is rebuilt, requiring the writing of custom processes.

Additionally, it includes an editor widget to aid in handling regeneration and addresses a bug related to undo actions, while also incorporating the addition of bushes for creating foliage mazes.

Guide Splines for PCG: Future Work and Takeaways

The team plans to improve the management and regeneration of PCG graphs, as well as work on more complex roof generation.

They also aim to enhance wall clutter with additional noise and industrial patterns using a plug-in.

The presentation highlighted the utility of sampling strategies, subgraphs, and data tables in the flexible Geometry Script, emphasizing the continued development of scriptable tools for PCG.