this can be used for a lot of different kinds of applications in the real world because often you don't just want detections on individual frames; you actually want to keep track of, like, where are your optics in the scene. So, this is really cool for that. Thank you guys for watching this video here, and again, remember the Subscribe button and Bell notification on that video. I also have these courses here that you can check out. I both have for OpenCV GPU, so how we can actually use OpenCV GPU. We cover some really nice things, all the different aspects within OpenCV GPU support. And then, I also have this course with the YOLO V7 model with architecture, how we can train it, how we can deploy it with OpenCV and ONNX and PyTorch. So, thank you guys about this video here, and I'll see you next one. Bye for now.

basically just have our model. We have our frame. We pass it into our model. We do a four pass in our model, and then we'll get the results out here at the end. Then we can split up the results here to our coordinates of our bounding box and also the labels for the detections that we're doing. So let's say that we wanted to take the person, then we're throwing out the boundary box or the corners of the bounding box around me, and then also a label, which will be the corresponding class that we're detecting. So that could be, for example, a person. Then we can also convert a class to a label. We can plot the boxes here as well. So if we have greater than some confidence score threshold, then we can act like both drawer, or we can actually just append our detections to a list, and then we can return that list, do something else after that. That is what we're going to do in this video here, and then we're going to draw our detections after they have been through our tracker.

running 30 frames per second, and we have a couple of frames, or we miss our detection because we have some occlusions, or we just don't detect it in the model, or if it's not larger than our threshold, the confidence score, then we might lose track, and then we'll just lose track of the optic completely. Where if you're using a trigger, we can actually track the exact same object, we can assign an ID to that object, and then we can track it over a number of frames, and then we can play around with some parameters and so on. But first of all, from the Subscribe button and Bell notification under the video here, only 10 of them do you guys watching this video here are actually subscribed to the channel. It's just a single click and helps me and your channel out in a massive way. So here, we're going to use the yolovice vibe model. I also have a course with the yellow V7 model. If you want to use that model instead of the old V5, it would actually be faster to run with the

you will need to go in and do your class labels, initialize your classes yourself, but we can also go in here and do, for example, like cup detection. So, you can just do that, and then it will detect cups in the image instead. But here, we're just going to go with person. Then, we're basically just returning the frame and also the detections. So, this is the whole YOLO V5 model detector. Again, you can use a custom model or a printed model. You can even go in and use the YOLOv7 model if you've been through my course. Here, we're just going to open up a video capture. So, we're going to open up the webcam. First of all, we set the webcam resolution here. So, it is 1280x720. So, this is basically just the resolution of our webcam. We initialize our detector here, which is equal to YOLO detector. The model name is equal to none because we're just going to use a pre-trained model detecting people in the frame. Then, we're going to import

because right now, we're not just doing detections; we're also doing tracking. So, we need to keep track of specific objects. So, if you just have a boundary box and we just like to have a detection here, miss a couple of frames, we have a detection here, we can't really keep track of the ID. We don't really know, like, what is the exact ID. So, this is more like tracking optics around than just doing, like, object tracking and getting some values of where the update in this single frame here. So, again, we plot our boundary box, and we also plot our ID on top of our boundary box so we can keep track of the correct boundary box. You can also use that for, like, indexing different kinds of monitoring boxes. If you don't want to know, like, how's people moving in the frame, then you need to actually keep track of the ID. Also, if you're tracking some vehicles, if you have, like, some attendance system or something like that, then you can actually have an ID or just want to know.

we're basically just returning our detections and also the image. We're not doing anything with the image right now, but you could actually go in and draw the detections. You can maybe draw the detections in one color and then detract objects in another color, and then you can see the performance and compare, like how does the Optics take their work with and without the tracker. So basically here, we just have our predictions. Then we can just pass that into our optic tracker. So this will be our tracks that it returns. So basically here, we have object tracker. We hit dot update tracks, then referring the detections, and we also throw in the image. Here we can see that the boundary box expects to be in a list of detections. So then we can basically just see here that in each tuple, we have this left-left top corner. We have the width and height. We have the confidence score and the detection class. So this is basically just a format that we need our detections to be in when referred into our object update tracks here. If you

ultralytics, and then have they have this repository YOLO V5. If we throw in a model name here, it will load the custom update detector or like the custom the only five model that you're trained. I have videos about like how we can train your own ulv5 model, and again the course about the only seven model. We're all over like the architecture of the model, like how does the model work, what's the advantages, and I also go over like the advantages over yellow V5. So definitely check that out if you're interested, but here you can either like you load a custom model or just a pre-trained uh you always YOLO V5 model here, and we're going to use the small model of this example here. If no model name here is passed into the function, we can also like have some score frame here. So this is basically the forward pass that we're going to do of our frame. So first of all, we need to resize our frame here to the correct format for July 5, and then we

and two consecutive frames before we actually initialize this as an object that we want to track over a number of frames. Then, if we miss this optic here for five frames, then it would look like delete that track. If you're not losing track within like five frames, you can like tune these brothers here for your own application. But if we are doing two consecutive detections, we will initialize an object every track, and then we just drag it until we lose track of it for four or five frames, and then we will act like track it really nicely around in the frame, as you're going to see later on when we run the program. We can also do some other different kind of parameters here. We won't really go into details with that. We have some like feature extractor, we have a neural network botjet, we have a metric here for like the matching cosine distance that we can set up, and also the non-maximal suppression overlap function value here as well. You can play around with that for your own application, but the most important ones

from our deep sort algorithm here. So basically, here we have our deep sort real-time. So first of all, you need to install that. So basically, we can just go in and pip install it. So here is just pip install, and then we just have this deep sort, and then we have this real-time one. So here again, my requirements are always satisfied, but you should go in and pip install this deep sort real-time module. Then we can just go inside that and import the Deep sort tracker. We can then initialize it here. So then you'll actually get a folder over here to the left where it basically just stores all these different things. So we have like a Kalman filter, we have like our linear assignment neural network matching. So we're using the neural network to do the matching, setting up some metrics for like how can we match the detections to each other. So that is why it's called the sort instead of just the sort algorithm that we have used in another video. We'll have this detection class, which is basically just initializing the

so here we're basically just taking out the coordinates for the top left corner and the bottom right corner. So, we're going to click drawer boundary box with that. We're not going to do that here because we're just going to do it after we're past our detections through our tracker here. We're just going to get the different kind of like format here that we need for our deep sort tracker. So, here for the default tracker, we actually need the top left corner, and then we also need the width and also the height. So, this is the format that we need to pass it into. And then here, the fourth item here in a row, a vector here, or like a row vector, or Matrix for our detection, it will be our confidence score. And then we can also pass out the label for our update that we have to take them. And this over here, we're just going to do person, but we can also change these to all the other different kind of like classes that we have inside of the cover data set if you're using your own data set.

Just go inside this function here. It acts like a hold for all the different kinds of algorithms within Deep Sort. It uses the Kalman filter and so on. First of all, we need to do some embeddings of our detections and so on. We do non-maxima suppression, so that is what we're doing here. Then we update our tracker. Before we update a tracker, we're actually like running this prediction step in our Kalman filter. After we have predicted our tracks, we will actually like run this update. So, if we don't have any detections, it will just do this prediction state. Then we would like to just use the track from our Kalman filter that is tracking the objects. But if we have some good detections that we want to use in our tracker, we would like to update our trial here with our detections. And then we just have this loop going over again where we just do a prediction step and an update state.

only 7 models and you will also get better performance and also show you how we can actually deploy it. Then you will get the boundary boxes out and then you can just directly use those detections and bounding boxes with the DeepSORT algorithm here for tracking your detections in the image. So definitely check that out. I also have quarters about how we can use Ohm's DV GPU support so we can actually improve the speed by a lot, like up to 30 times. Or if you want to go from a couple of frames per second to almost 30-40 frames per second in your algorithm, then you'll be able to create real-time applications here. I'm going to show you YOLOv5 detecting running on the GPU, and then we also use the updated tracker. It will run on the GPU as well, so this is actually going to run really, really fast. We're not going to see the results at the end, but here we're just going to jump straight into the code. We're importing the modules so

miss before we discard our track, but here I'm just going to move around. We can see how nicely it acts like tracks me around in the frame. You should again fine-tune the different kind of values. We can also try with another class here just to see how it works if we just move any cameras around instead. So basically here, we're just going to go up and change that to a cup, and then we're just going to hit save. And now we're going to run the program again. I'll take the webcam down so we can act like track a cup here instead. So here we can see the detection for the cop class. This is actually like a class, but it's under the cup class as well. But here we can see it just keeps track of it really nicely even though we move it around. Here we lost track of it. So this is the class. This is really hard to actually like track around in the image frame because again this is not really as good as just a cup. So here we see we

we're going to use OpenCV, NumPy, time, and also Torch. We're going to use PyTorch to actually load in our YOLO V5 optic detector. Then we have created a class here for the YOLO detector. I have videos about how we actually initialize this class here, and then we created a class in another video, so may definitely like check that out if you want to know more details. Here we're just briefly going to go over it so we can talk about the Deep SORT tracker afterwards. So here, first of all, we just have our initialization function, which is for the model name, if you want to do it on a custom optic detector, or we can just use the pre-trained one of the pre-trained YOLO model as well. Here we're just going to set the class names, we're going to set CUDA here if it's available, or else we're just going to run our model on the CPU. Then we're going to load our model, so we use Torch, PyTorch, to load our model. Then we're going to inside

detections with the boundary box class, like class confidence score and so on. We have a Tracker, and then we also keep track of our tracks, so we can actually go in and set our tracks when we're running our tracker, because inside of the tracks would actually like be detract boundary boxes from our YOLO detector here. First of all, we need to set up some different kind of things, so these them just like default values, like all these values down here, or default. I just want to show you like what are the different kind of like parameters that we can initialize our deep sword tracker with. So max edge here is probably like one of the most important ones. So this is like how many frames the real live, the loudest tracker here to miss before we just discard that track or that boundary box or detection that we're like detecting and also tracking in the frame here. We have an init, which is basically just the number of detections that we need before we initializing a track or a detection. So basically, we're doing, we need to do two detections.

and step for our Kalman filter. And then basically, the Kalman filter just tracks these detections and bounding boxes over a number of frames. So, that's basically like the idea behind Deep SORT. It's basically just this sorting algorithm running like a Kalman filter on the hood, and then we use some deep learning metrics to try to keep track of the matches, always matching the bounding boxes and detections with each other. So, basically, here it just returns the track. So, this will be all the tracked bounding boxes from our detections. Then we just have a for loop running through all the tracks. We get the track ID, and then we also get the bounding boxes here. So, then we just convert our LTRP to our bounding box. So, this will be our top left corner and the bottom right corner. Then we can just draw a rectangle and also put the text. So, we're going to draw a bounding box around that with OpenCV on our image, and we're also going to put the ID on top of our bounding box.

like how are people moving in the frame. We end our timer here and then we just calculate the number of frames per second that we get. We're just using intro if it escape on a keyboard. We will turn into the program. We will release our webcam and destroy all the windows that we have opened up with opencv. So now we're going to run the program here and I can like see the results with this update detection and tracker. So here it's just opening up the program. Now we can see the frame here. We're detecting one person inside the frame. So here we have around like 60 frames per seconds on my TPU. And again we assign an ID on top of the boundary box. So here we just keep track of me. We don't lose track of me at all in the frame even though I'm moving around here. We lost track. You can see for five frames probably. It should actually like tune that threshold because we're running like 60 frames per second. So it's only like a couple of frames that we need to

So, in this video here, we're going to update the tracking with Yellow V5. So, we're going to have a Yellow V5 model running for optic detection, and then we're going to use the Deep Sort algorithm to do tracking of those detected objects. So, this is actually like a really cool video, like in previous videos where I've been talking about how we can do optic detection and so on, but now we're going to add tracking on top of our optic detector. Because when we're detecting optics, we're just doing it from frame to frame. So, we just pass in a frame to our model, we get a detection out, and then we just get the detections out. So, if we're not doing detections every, for example, like running on a live webcam, then maybe we'll miss some detection from frame to frame. But if we have a Tracker, we can actually track our bounding boxes and our detections over time, and then we can actually keep track of our optics even though we lose detections from frame to frame. So, if we're, for example, like

keep track of it. It is ID19. Even though I'll move the camera around here again, is it true in the threshold values? I can take up another cup here again. We just snap on it directly. We can move the camera around. We still keep track of the object. So again, this is a really nice optic detector and a really nice tracker. It can be used for a lot of different kinds of applications. And definitely, like if you're using up detection, you want to keep track of the IDs. You want to like get that information as well. Then it's really good to use the tracker and used in most real-life applications because this is just really cool. Even though we have some inclusions, we can even still like keep track of it. You can see here the cup. Maybe the detection lost track, but we can still act like detected. And if you have a static environment and you want to have like, for example, like occlusion, occlusion is coming in front of your camera. So maybe like people walking in front of the camera in some

are the max H and the in in it. So here, basically, we can just go down, and again, we can open up our webcam. So now, we're basically just going to open up our webcam. Here, we're going to read in a frame from a webcam. We start our timers so we can time how many frames per second that we're actually like get with running this YOLO update detector plus tracking device optics around with the Deep sword algorithm. So here, we're basically just going to do a detection. We just have our detector.score frame. We pass in the image. We do a forward pass, and then we get the results out. So this will be the results. Then here, we can call this detector.plot boxes. We have our results. We have our image, and then we basically just pass in the height and the width here, and also the confidence score. So we can actually like um rescale our image. We can set in the confidence score. So if it's above a confidence score, we will act like save our detections, and then at the end here.

kind of like um Warehouse environment. Then you can basically just set up like the maximum maximum number of images that can that can be missed. You can just set that to a really high value. And then maybe like even if you have occlusions for a couple of seconds. They appear in your setup. Then you can still keep try every update behind that occlusion or like behind that person walking in front of your camera. So this is also one of the advantages of using a Tracker together with an object detector. If I just take the camera up here. We can see that just running this update here. We run around like 130 40 frames per seconds on my tpq. And then when we act like doing our tracking. We're down to around 70 maybe 80 around 70 frames per second. But this is still really fast by getting these Avengers here off the tracker. Where we're tracking around the Optics in the image frame instead of just doing detections on individual frames. So that's pretty much it for this video here guys. It's been really cool.

Object Tracking with YOLOv5 and DeepSORT in OpenCV

Welcome to my AI and Computer Vision Channel! We are going over some cool things on this channel with main focus on hardcore AI, ML, and Computer Vision. I'm doing both theoretical and project based videos. Visit my AI Career Program at: www.nicolai-nielsen.com/aicareer

Object Tracking with YOLOv5 and DeepSORT in OpenCV

Object Tracking with YOLOv5 and DeepSORT in OpenCV

스크립트