Implementing Hearing Perception in NPC Controller

The NPC's behavior is modified to react to player noise by playing a sound and reporting it to the perception system.

### Implementing Hearing Perception in Unreal Engine for NPC Behavior
* 🔊 Create a new C++ file to add code to the player that makes a **noise** and reports it to the **perception system**.
* 🏷 Define a new **tag string** in a header file to be used for the **sound stimulus**.
* 🎮 Implement a function called `OnDistract` that **plays a sound** when the player makes a noise and **reports the event to the perception system**.
* 🎚 Expose a property to set the **sound to be played** in the editor.
* 🧠 Modify the AI controller to process the perception events, and add a **private array of actors** to store the **perceived noises**.
* 🔍 Iterate through the perceived actors in the `OnUpdated` function of the perception system to check for the **sound stimulus tag**.
* 🧠 When the sound stimulus is detected, **update the blackboard with the location of the stimulus** and **set a flag based on whether the stimulus was successfully sensed**.
* 🌳 Implement a new behavior tree task to **stop investigating** when the NPC has **investigated the sound stimulus**.

it's time to do some hearing perception. The idea is to add code to the player to make a noise and to report that to the perception system. Looks are then out here in perception study to the NPC controller and modify the behavior tree to suit. We need a new file cuz I need a tag string. I could put it in blackboard keys off not going to cuz it's not a blackboard key so we had a new ID son want a header file go get this location right. I'm gonna call it ion tags. Actually, namespace tags header includes so we go to the plaque. It's a header so we need a function in here which is called when the player makes a noise. It's gonna have a button on the gamepad and key on a keyboard and possibly a mouse button which player noise and report to the perception system. I call it on distract. I'll implement this. I go to here. I got do a bind action we shall have tried later in unreal. I call it distract and went to go to on distract. Oh yes, we need a sound to play.

and for that, we're going to expose you property that would be somebody set it in the editor so that's that. Yeah, I need a header for this but some of it my first for you sound nice stretch sound think I'll make the category sound not animation and are expected to complain about of it's not popping would normally need everything could are any other headers but that header is normally where you sound basis. Oh, it actually is need some heading closer we don't have so duplicate this one make it hearing need a tax hey sigh. Oh, it sags and some morning when you gang place tank so you don't think we've got. I think we need engine as well god you do this function if we've got but to stretch sound okay from ekta location. So, Inge you gain high stakes we play the sound by sound location need the world the sound play the location then we need to report this event to the perception system report noise event and you pass the world again liar character are the maximum range.

which I think if you see always it's no believe me and the tag no he's tagged and now we need the AI controller go ahead. The perception system it's a fact so it can hear what's happened so we need to add something to the private section do you think in fact. [Applause] and we're gonna need a function which is allocated delegate calls and you have to have the you function Mac Romney's also it don't work Boyd on updated likes it's a array of actors pointers to I hacked old doors now and CP hey Hulk implement this first there in the CP got some ahead destroyed I got some extra here just had IR tax office copy that make that human the gameplay steaks now we need to set it up first now I'm gonna make a operation here. I'm gonna put a lot of this inside it and if statement because this create default sub watch it could file so I do yeah so right side config we do this lot I'm also tied in this code up it give it.

it do a multiple assignment like this because it'll set to choice. I couldn't so it's that and we want hearing you Hisense come back here in a little name what set the hearing range set it to say 3 thousand. You say it to like that work of course copy most of this so a bit typing Isis. I don't that change this delicate chase delica is a perception out lazy strangely word term mind if you ask me do the code for the actual function this on update it needs and it got gonna use a cup an ST forums so they get the perception in before of each one of these as we go round around the exception. So what that does sigit specific opponent to get the actor's perception on the actor and puts it in him before me now. I use use a for loop on that. I'm gonna make this guy 16 Malloy. I've got the stimulus that this actor create it so I'm gonna look for this tag. If I found the tag then he's a

human stimulus. If not, it is quite possibly a psych one. So every secret to the noise you get the black ball. Oh, should be a comparison operation of Simon. So get a blackboard. Yes. Ah, yes. If we don't have this, we need to add a blackboard key to that. I'll do that in a minute. This accessory, since I need to write the location that's without the stimulus location machine. The effect would be the player cause the reason why we've got this if statement there. Now its clause is because when he goes to investigate the sound. When he gets there, he could find the ply on which case this will happen this branch here and it will set can see by work depending whether it the stimulus was successfully sensed. So blackboard keys, we need a new string is investigating. Stop complaining now. No any new tasks. No behavior tree task called stop investigating which switches off this is investigating. Okay. So I'm going to use my new method. Okay. Got a fix of includes the controller for the NPC blackboard.

component and a black keys file give thee no to name such as in here execute task and we just use that to access the blackboard such it get it. That's really it so now we need to go. Well, well guess we would build that usually and I'll see you in unreal engine building successful do that input binding first before forget. I wonder what doesn't work. I'll make a new action mapping call it distract heads three of them first month we need gamepad. Hey um bye keep on set seed inside and then the mouse now to go for right mouse button left mouse button attack. So what we need is a new sequence node side track one off. I call it just investigating believe. Now this doesn't work do it anyways is what was done in the blueprint version and he used but tight to face BB entry to make the character face the source Allah noise the BB entries target location ie waiting once. I'm gonna give it one second lot then I move to isn't it to target location then another wait make that one second then this some stop.

investigating. That's the first bit we need to decorate on this, which will be not target location but ah yes. Go ahead don't in C++ Paco. I don't know key called he's investigating tray spacing also that set. There's a number of issues with this. First one is a bug to be that sort. Partly works. It's let it go oh yeah. Got said sound every upslope make it collapse one like an explosion actually that off for a minute. See it actually but what happens as he freezes. And a bit of investigation showed that is this. This some note here this rotate to face BB entry conflicts with the NPC's. But it's in the MPC. Yeah in the character movement. Many moons ago we changed the character movement rotation settings to orient rotation to move. And that seems to conflict with that node. Now you can either switch it to the other one which is used controller desired rotation. You got one of them on because otherwise he it doesn't pro Tate when it moves. These pucks weird. My solution is to just

delete this notes and I really see the point anyway it's gonna move to the player is right so first of all he's walking. I would like him to run towards us. The other thing is that when it gets to us and Caesar she doesn't immediately attack just wait for the sub-tree to stop and then does attacking. So we go back to the tree first thing it affects us of speed so I'm gonna put that service on the pedro' change speed. I like it kick it him not make it yeah make it six under. Oh the thing is this one myself five hundred that's madness one 502 then so may not 500 so she should combine you know if I'm gonna do this noise yeah next so to fix this thing about no immediate aking what we do is oh I need to make this yeah I've gotta set the observer box on that decorate both. I got another decorate had another decorator which they can't see set it to kandacy liar and make it a thought now what'll happen.

is if he when he gets to location if he can see the plier. This is investigating night will be a ball winning. It will come back and go down to here and start attacking and chasing stuff, which is what we want. If I can rearrange these no too big, so get a bit out and let you see the tree. Me thinks it will work now, cept it stopped working. Why was that? There was a reason, but yeah, well I managed to my brokenness now. No, it should be is not set, I believe. Yeah, there comes over and then he immediately attacks because now can see pie is setting a ball up sub tree. Yeah, I got just got a logic wrong way around. I can lose him can be a bit difficult. He's gone back now. If I make us make a sound, what happened so they NPC maybe is you've gone back to patrolling. What happens to be sees us again, yep. So I think it's working, that's it as basics of hearing perception.

UE4 C++ AI Series - Part 10 Hearing Perception

MrCxx

Adding Noise Perception to the NPC Controller

A new file is needed to create a tag string for the perception system.

The code should include a function to report when the player makes a noise, with the ability to play a sound and report the event to the perception system.

The function should also include the necessary headers for sound and location, as well as the world and the character making the noise.

Revising AI Controller Functions for Actor's Perception

The AI controller needs to have a function for allocated delegate calls and updating likes for the perception system.

The code should include setting the hearing range, implementing the perception update function, and using a for loop to iterate through the stimuli.

The function is designed to identify specific opponents based on the actor's perception and the stimulus created.

Creating a Blackboard Key for Stimulus Location in Unreal Engine

A new blackboard key is needed to track the stimulus location for the NPC's behavior.

The key will allow the character to investigate the source of the stimulus, with a new method using the controller for the NPC blackboard component.

Input binding for the action mapping, including gamepad and mouse buttons, is also necessary for the functionality to work correctly in the Unreal Engine.

Investigating NPC Behavior and Character Movement in C++

The NPC behavior is causing issues with rotation and movement, potentially leading to freezing and collapse in the game.

The conflict seems to arise from the character movement rotation settings, and a possible solution is to switch to the "use controller desired rotation" option.

Additionally, there is a need to adjust the NPC's speed and attacking behavior, possibly by setting up an observer box and a decorator for the attack action.

The speaker is trying to adjust the NPC's behavior to make it more responsive to seeing the player.

They are experimenting with the NPC's actions and trying to make it work effectively.

The speaker is also considering the basics of hearing perception for the NPC.