All right. Sorry for the delay, everyone. We're having some technical difficulties with getting everybody all set up on this side. Sorry about that delay, but we're ready to go. Okay. Thank you for joining us today. My name is Jeffrey, and I will be your host for today's webinar. Today, we're going to be talking about maximizing video AI performance. So we have a guest here today from Opera. We've got Mahdi on with us. We're excited to hear from him. And we'll also be hearing from Ryan on the AMD side. So I'm not going to waste too much time. Just want to give a couple of pointers about the interface before we get going. On the right-hand side, you will see a questions box and a survey box. In the questions, you can drop any questions you have, any Q&A, and we'll try to get to you at the end of the webinar. And then in the survey, if you could fill out that

survey for us before leaving here today, that would help us out greatly. And don't hold the delay too hard against us on there. There is a couple of different modules up above. You should see engage and resources. If you go into resources, you will find a bunch of links. So you can run the demo we're going to run today. You can access all the GitHub information. You can go to buy the board that we're going to be talking about. A lot of different things in the links. And then in the documents, you will be able to download a PDF copy of today's slides. You will be able to come back and watch this on demand anytime. So you can use the same link that brought you here, and we'll also send a thank you link shortly after the presentation today. All right. I think that's going to do it for me. With that, I'm going to hand the ball over to Ryan to get us going. Okay. Welcome, everyone, to today's webinar. My name is Ryan Radjoby, and I am the systems

architect for video analytics in the AMD AI group. And I'm joined with Maddy Godse, director of engineering at Opera Technologies. Welcome to our webinar titled Maximizing Video AI Performance. So we're going to show you how to use VMSS, which is a powerful framework for building video AI workloads for live video analytics on AMD hardware. Thank you, Ryan. Happy to be here. So here's a view of our agenda. We're going to spend the first few minutes introducing the problem of video analytics and give a quick plug of the AMD developer contest and an explanation of VMSS features. Then we'll spend the rest of the webinar doing a hands-on tutorial showing you how to quickly install and run your first example in just a few minutes of VMSS on some AMD hardware. And then we'll get into a deep dive tutorial viewing the code and run a live demo of multiple pipelines created and run within VMSS. So just a quick definition, what video analytics is. It's really just the technology enabling

automatic extraction and analysis, and comprehension of useful information from live video sources or recorded video. It's going to use advanced machine learning techniques, usually running convolutional neural networks to identify, categorize, and track in real-time objects in a video stream. And so the common use cases are in traffic analysis for a smart city scenario, security, as in counting people or crowds or facial recognition, and then there's a lot of applications in retail that are for self-checkout, kiosks, identifying objects and classifying them, or finding ways to curate more, better shopping experiences for customers in retail. So here's our quick plug for the AMD Pervasive AI Developer Contest. And this part of this webinar was geared to educate and inform some of the contestants in this contest. So AMD's given a challenge for developers to create a new platform for their customers, and that's what we're going to do today. So I'm going to go ahead and show you a little bit of what we're going to do today.

So AMD's given a challenge for developers to create innovative projects that run on select AMD hardware. One of the categories was a robotics AI competition using the CREA KR260 Robotics Starter Kit. So what that is, this is a system-on module developed by AMD that has a lot of IOs for doing robotics applications and vision applications, as well as the compute hardware for running neural networks very efficiently. And so this project is in, this contest has already commenced, and we're hoping that contestants are interested in using VMSS for their contest. So a little bit about, I'll just give a brief preview of what Opera's VMSS 2.0 is, and then before I let Matty get into some of the more details. So VMSS stands for Video and Machine Learning Streaming Server, is a video analytics framework that unlocks the power of AMD hardware to create complex computer vision applications. VMSS is two things, it's both very easy to use and also very powerful, highly customizable. So we're going to demonstrate running the first

examples with no code at all. So that'll show how easy it is to use. And then we'll demonstrate the power and flexibility of VMSS in our deep dive demo, where we're going to make modifications. You're going to see different applications built out by modifying the code in there. And VMSS has been made available for free to contestants of the AMD Pervasive AI Contest. So now I'm going to pass it on to Matty. Go ahead, Matty. Thank you, Ryan. First of all, I'm super excited to be here. But along with me, I have a few of my teammates that you can ask questions during this webinar, and they will answer during the chat, so feel free to drop your questions during this webinar. We have Matty Zainali, our software architect, along with a few other engineers, Matthew and Emu, that can answer your questions during this webinar. So as Ryan mentioned, we've made this software free for you guys, and this is the first time we're doing this.

And our goal is to allow you to be able to build the coolest applications for this contest coming up. So, with that in mind, we tried to make it as easy as possible to install. It's just a single command that we'll show you later that you could just basically get your Ubuntu setup, and then run the single command, and get everything set up. It will install packages such as Docker and get your Docker container ready for you. We use this framework for our own needs of building applications for different customers. So you can definitely use it the same way to create real AI challenges that you have for your contest and create the application that you need. There's quite a few different ways that you can interact with VMSS. One of them is through the web application that will show you quickly how you can just run an application with no code. You can actually modify that for your own need with a few parameter changes. For example, we have a YouTube video on our GitHub page that shows how

I go through an example of doing a car counting, and then just switch the model by modifying a couple parameters and turn that car counting application into a person counting. In terms of customizability, we tried to make it as easy as possible for you to implement your own logic. It's all written in C++, but there is a script that will get you started by creating a boilerplate for your code, and you can integrate your logic. Within the same Docker, you can actually build your code and then test it and integrate it into our framework. VMSS is basically a graph filled with a bunch of nodes. We have quite a few nodes available for you for free that you can use. The code for most of the nodes is either available or common, so we basically have almost everything in terms of the nodes open source for you to be able to use or modify or contribute to our project. Finally, in terms of the AI inference, you are actually enabled to use a lot of these AI models available in AMD Model Zoo using our inference nodes.

as detector node, or classification node, segmentation node. There's quite a few of these nodes available that basically we've taken care of all the inference code for you. All you have to do is just give it the name of the model that you want to use and the type of the model. Within two or three parameters, you can actually go through almost all the models available in AMD Model Zoo for your pipeline and test it on different videos. In the next slide, if you, I guess, is it, okay, there we go. VMSS 2.0, you might ask why we call it VMSS 2.0. We had VMSS originally passed to us by Xilinx. We also had quite a few applications that were being implemented at the time along with maintaining VMSS, but through our pains, we gathered all the issues and all the enhancements we want to have in our next release of VMSS, and we created VMSS 2.0. By doing so, we actually were enabled to save us a lot of time.

not writing the same code over and over and making it into a modular design that you can actually reuse. A bunch of nodes and spend less time on video processing or common pre-processing, post-processing, running different AI models, and just focus on the logic that you need to customize. So we're providing that to you here as well so you can use. Some of the success stories of Apparel with VMSS is listed here. We have built face recognition application that we passed for Smart City project for Tencent. We have built, for example, a person counting for a big exhibition, Shanghai exhibition, and they've used that for basically going through the number of people entering the venue. And finally, our flagship application, which is our license plate recognition or vehicle management, so it does more than just license plate recognition, is currently deployed in quite a few parking lots in Singapore. Our partner SunSingapore is currently using our AI analytics software, which includes our own custom train model for this application that they're

using for collecting fares and managing their parking lots overall, which means that we are able to actually create any sort of applications. We have our own AI team, we have our own software team, web team, FPGA team, so we are quite capable of building custom applications for any of you guys who are interested to have us help you with your AI needs. So feel free to contact us and we'll be able to take it from there. I believe, Ryan, we want to have basically the next slide, which is how to install and then the web application we built for people. So for getting started with VMSS 2.0, it is quite simple, as I mentioned. All you have to do is follow the guide that we have on GitHub. It is quite simple, as I mentioned. All you have to do is follow the guide that we have on GitHub. Ryan will show that shortly that you have a setup page that you guys can basically go download Ubuntu from AMD website and once you have that downloaded and flashed onto your SD card.

then it's a single command that will take care of all the installation for you. It will take maybe a few minutes to install, pull the docker image, and then create a docker container for you. But once that's ready, you're good to go. First, we're going to show you the web application. You can quickly just add your kriya sound device by creating a name and adding a serial number. Once you have that ready, you can save it, and it will be available for you. Then you can actually use any sort of RTSP stream or a camera by just creating the name for it and then copying your URL that you'll get from your camera or online and save it. Then you can do a sanity check, make sure the video is playing. Once you see a stream such as this one, which is a sample video we put there, you can go ahead with the AI task. Here you can just create any AI task you want. You're going to do a custom pipeline.

select the camera that we just added, which is the demo face, and then next would be our AI host, which is our kriya sound that we added. Here you have our pipeline library, which is a collection of different examples we have for you. We have categorized them. You can search by label or application. It's quite powerful. You can put anything in the search box and you'll find it. We start with the detection example here. We're going to go with a person head. The first model you see here is a dense box. There's a face detection for you. Once you select that, you can see on the right side you have the graph configuration, and on the left side is a visualization of the graph. On the right side, you can actually modify it, and it will be reflected instantly on the left. Let's start it. If the model is not available, it will download it for you to your box, and then you can start the task immediately. Once you have the task running, you can actually

watch the results through our web app if the encoder is supported, or you can just grab the URL and paste it into your video player. We're going to use BOC here. You can copy it here and play it. As you can see, within a few seconds, you can have your video pipeline running. You can visualize the results. For you to use this web app, you just have to sign up for an account. It's a free account. You don't have to pay for it. As long as your query asylum is connected to the internet, it can recognize your query asylum and can do the same thing and play around with it. This is the application. As you can see, it's running the face detection that we just started. Let's go ahead and stop the task. It will remove it from the task lists. That is basically how simply within a few clicks you can actually go through our web app and run our examples. You can also select basically any of the AMD models.

Zoom models available with just a few clicks and run them immediately. That concludes the very short snippet that we had for our web application. I'm going to pass it back to Ryan to do a live demo of different examples that we prepared for you. Back to you, Ryan. I'm going to share my screen now. All right. Looks like I'm sharing my screen. Let me know if I'm not, otherwise. Thank you, Maddy. Now that you've seen how easy it is to use AMD hardware for your vision applications using VMSS's web GUI, let's look into the power and flexibility of this environment where we're going to go and modify the pipelines. You saw a little snippet of the graph description on the right that was at some of the code. We're going to go and modify some of that code and really make these simple modifications to have it work out for different application in different use cases. This tutorial that I'm showing on the left is all available on GitHub right now. In theory,

you can all follow along with me and run these examples just as easily as I will right here. We're going to get started. A video pipeline is constructed by determining your desired input and output media, followed by the logic and AI algorithms, and additional post-processing. VMSS offers a wide range of input and output options. This diagram here in the middle demonstrates the flow of data from left to right. On the left side is the front-end input sources that can be images or RTSP live video streams or recorded video files or even a USB camera. Those are your input sources that get then fed into the right into your logic application. After some sort of pre-processing that brings those into a sequence of image frames, VMSS provides a variety of nodes like detector, classifier, segmenter, landmark predictor, and tracker. These can also be run just as single nodes or as a cascade of multiple nodes, such as a detector fed into a tracker to a classifier, then to the next optional stage.

of post-processing those. You may want to draw bounding boxes and regions of interest, or labels on top of those inference results. Then it's fed to your output stage, which could be in the form of encoded video to a video file, or a live video stream, or saving images, or it could just be to take that metadata and produce things like notifications to SMS or email, or just writing the metadata to a database, which is really the most common form of outputting after live video inferencing. Our first example I'm going to get started on is going to show how we do. We're going to take a video file and run person detection using that video file. The first file is called RTSP person detect, and if we look at this, I'm just going to set some of the parameters. So what I'm setting here at the beginning is the input video source, so the path is to a video file that's on disk, and the next stage is into a detector, a visualizer, and then at the bottom I need to make sure I set the output path, which I've

already done so. I'm using an RTSP server and setting the name of the stream for the output. And let's go ahead and run this. So as I launch that and takes a little bit, you'll find all the instructions in this tutorial on the github. We'll talk about how you can set these different parameters and what the command means. I'm running the avaster binary for launching an application and I'm passing it the configuration file, so this is pb text format that is defining the graph that we're running. So we can see that this is running and I should be able to open up this video stream and open this up. And so as we've this diagram is going to show what we're doing in this pipeline. We have our video source at the next stage of a detector and those inference results are passed into a visualizer along with the original video input image. And so we're drawing on the results here and I actually see I have some parameters set in it that I didn't want.

set so it's not running detections on every frame. It's skipping. Okay, and so what we're seeing there is the original input video is fed into a detector and the results of the detector are just the region of interest, the bounding boxes of that detection. And I've fallen a little out of sequence here but okay, this is the correct configuration. So the region of interest is drawn on top, is passed into a visualizer, the box visualizer, along with the original input frame. And then those are going to be superimposed and created into a video stream that's passed out as an RTSP video stream. So as this comes up, we should now see person detection. So we're seeing bounding boxes drawn around all the people in this video feed. So that's great. Now I'm going to move on to the next portion of this and we're going to change this from person detection to face detection. And you can see how simple it is to change it from person detection.

first I'm going to change the input stream so instead of using that video file, we're going to use an RTSP video stream and do run the person detection example. So in our first example, we use a video file as our input. Now we're using an RTSP video stream as our input to this pipeline, and so we should see the same results because we're still doing the same person detection. I haven't changed it to face detection yet. And as this pipeline spins up, we should be seeing results coming through. So in the second here, okay, so the results are the same because we're using the same type of sources, but rather than coming from a video file, we're seeing it from RTSP live video streaming protocol. So now let's modify this description. Instead of using of running face detection, person detection, we're going to modify it. So I'm just going to comment out this line, which is defining which model is running. And instead, we're going to use this dense box 320 by 320 facial detection network model. And so we just need to specify that we're running a

different model, and we also need to, because this is a, this has a different type of detection result. We change that parameter as well, and we can leave everything else the same. So, this should run, this should produce a, a new stream of, of output results with just the bounding box drawn around the faces that are detected. And so, you can, you can go through, and you can modify these. If you want to walk through this tutorial yourself, you're, you're free to go and modify the code. Or we've also added links or just files in the tutorial for these different stages of the application. So, you don't have to modify the code if you just want to run the person detection without any modification. And it takes a little bit of time for the, the stream to build up. But once it has face recognition or face detection and bounding boxes are drawn over these faces in that video stream. So, this is great. Now, the next, next phase of this tutorial, we want to show, is, is for changing a detection interval and adding a tracker. And why you want to, why you may want to do that is

so what we're going to do here is change this detection interval to run on every third frame. You may have a model that is too computationally expensive and it can't realistically be run for every single frame of video. So we're going to say run on every third frame. What you'll see from this is we're just skipping every two of the frames and will not run the detector. We'll just run that on one out of every three frames. That means that the region of interest in the bounding boxes will only show for the detection that's made. So every third frame should have a detection face drawn on it. The effect of that is a little bit of inaccuracy, and you'll notice it when these frames come up. There will be a flickering because every third frame is drawing on this region of interest. This is great for reducing the compute load on your device, but it doesn't make for a great result. So the best

the best way to work around this is to add a tracker. In order to add a tracker, I'm going to drop in this bit of code, and then I'll explain it a little bit while the tracker is while this pipeline is building up. So, I'm going to insert the tracker between the detector and the visualizer, and then I just need to change the input of the visualizer because it's no longer taking results from the detector, but it's taking the output of its input stream. The output of the tracker. We launch that while that's launching. This diagram shows what we've done. So, we've taken what was a video source to the detector to a visualizer and to a video output, and we've inserted our tracker. So, we've dropped that right in here, and what that's going to do is it's going to follow those detections that are only running every third frame, and it's going to essentially fill in the detection results without having to run inference using a detector.

which is a much more expensive neural network running on there rather than just running the tracker in software. What we see is a smoothing out and a connecting, sort of connecting of the dots from frame to frame, and you get very accurate similar results as you would if you render detection on every single frame. So this is a great way of either running a much heavier neural network on your hardware or as a way of you could in theory reduce the frequency that you're running your detections and run multiple concurrent streams and pipelines on the same device using this method. Okay, so now we've done done the tracker. What I'm going to add now is I'm going to change our input from an rtsb video stream instead to use a usb camera. So in order to do that, we're going to drop in this chunk of this chunk of text, and I'll go over this a little bit too. So if I go into modify this, what I'd like to do here is now I'm going to comment out the portion from the visualizer down to the output sync.

and instead, I'm going to insert that block. Now, I need to change the output path, the IP, and port of my RTSB server. So what we've done here is fill in using the visualizer video filter. It's going to set to a new output resolution. We're going to encode that to H.264, and we're going to send that out through the computer. And we're going to send that out to the computer. And we're going to send that out through. Oh, I also need to insert the at the input. So instead of using our regular video source, I need to add in the USB camera. So, I'll comment out the original video source and add in the code for a USB camera that I have connected to the creason on. So if we want to launch that now. So as I said, what we've done is we've used the USB camera as our input node, and we've changed the visualizer to take the results from the tracker, which it was doing before. And then, um, those outputs go to the video filter and encode that to live video stream. And if we

we should see this come up here any second. Okay, and so you're seeing this result from the USB camera connected to the Creason right here, and it's running the same network, the face detection with a tracker. Um, whether I'd say no equally good results. So, I think, um, now I'd like to stop here and pass this on to Mattie who's going to show the next stage of this tutorial, which is where you can tie in, um, a notification node and, um, to create some SMS text messages based on the results there. So, um, Jeffrey, if you can, you go ahead and play the video, the next video. All right, great. Um, so let's just talk about, uh, what we're going to do next, which is integrating SMS notification to the pipeline. Uh, think of a scenario that you have a person showing up at your door, and you want to make sure you're notified somehow that the person is there. So, that's why we have SMS notification. All you have to do is, uh, at the bottom of the examples that

ryan showed you can just add the two nodes to json and um notification message. Which we're just doing right here. Once you add them, uh you just need to go configure them based on, uh, your own, um, fields that you get from, uh, Twilio. So we're going to go over it right now. So to json just transits your notifications or metadata into a json format. Next thing you need to do is fill, fill out these information about your sender receiver. So how do you get that? Your receiver is basically your own phone number that you want to use, um. So for the receiver, we're going to use a free app called TextNow, which gives you a free phone number that we're going to use here, um, and use that as our receiving text messages, um, app. Once you filled out information, then you'll have your sender and send a username, password, and url. Those informations are available on Twilio. Once you get your free account, uh, you will be provided these information. You just need to basically fill them out, uh. We have a little snippet on our GitHub that shows how to get those values next.

would be your uh basically manual track options. So in manual track options, is basically a filter that you determine, uh the filters you want to use to be able to, uh filter out what to send. So here, we're using, uh the region of the image and also the age of the track. So once you have all that filled out and you put the task ID in the bottom, you are basically good to go. We use, uh our own the same way that Ryan was running the pipelines. Add us here dash c, and you pass it the, you know, the pb text graph that you're using. Once you run this, uh you can also view the results at the same time. So, um, on the left side, we're going to pull up our VLC to show, um, the live video of, um, what the results look like. The path where it would be provided in the video sync, uh you just copy that, put on VLC. On the right side, we'll have our SMS, um, app which is TextNow to receive these notifications. Um, so the filter we put for this was about, I believe about 30 seconds of constant.

detection of a face um so I believe at any second uh with the this person performing tai chi you'll have uh the notification sent your text message. Once we hit that threshold, which should be around nowish and there you go. So you have your text message of um there's a person in the video stream that you selected. So to kind of wrap up um what we've done so far was we showed how you can just start from a single um example with you know a video file as your input and rtsp stream as your output. You can basically start there, just keep modifying the same file, change your input, change your output, change the model um and go all the way to how do you want to consume the results. So we've showed how you can visualize it here. It was how to get a text message from the results. Obviously, there are many different ways that you could get your results consumed. Text message is only one of the ways you could do. You can build your own custom way of using it. You can use some of our other notification methods that we have available.

As long as you go over our GitHub repo, you can basically cover everything. Um, that basically concludes, I guess, our demo section of this webinar. Um, we can now pass it back to Ryan, um, and go where, I guess, the next slide that we have, which I think should be the Q and A. Okay, yeah, thank you, Maddie. Yeah, that's great. I think, um, real takeaway for that is I hope, um, users of this robotics kit, if you are part of the Pervasive AI Developer Contest, you can see how you can really tie this into making a kind of a larger system application. So that's pretty neat, tying that into to use this notification node and produce SMS text messages based on those inference results. So, in summary, I just want to wrap up what we've shown. So, we've shown how to do like some really quick, no-code examples using the really powerful VMSS GUI web GUI. So that can that can connect to your device, you can manage it from the web interface, and run many different applications using the different models from the AMD.

model zoo on that device um and then we've shown how you can get into the code, the graph definition, and make modifications. So using different input sources video file live video stream using a webcam and how easy it is to change your inference stage to use different types of inference models and output to different sources as well. And if you want to incorporate a tracker, there are options for doing that. And if you want to see some more examples for cascading networks and adding classifiers on top of it or doing adding some more custom business logic based on the inference results, there are examples in the github. The documentation is fantastic. So what I'd like to say is if you have hardware, please try out this tutorial and give us some feedback on it as well. It'd be great if you have if you have a real a real commercial use case and you're interested in incorporating amd or vmss into your computer vision application, please reach out to me or to matthew by email or on linkedin and we

we'd love to discuss, you know, how to assist you in commercializing your computer vision application. And if you're looking for some support using VMSS 2.0, please post an issue on the VMSS 2.0 GitHub. There should be links in the webinar that Jeffrey has posted for links to the documentation, links to the GitHub, the tutorial steps. So thank you very much. At this point, let's open it up to question and answer. So I can click over to the next page. Thank you. Thank you very much, everyone. Thank you, everybody. Yeah, awesome. Thanks, everyone. Yeah. So Ryan is correct. In the right-hand side, you'll find the resource list. In there, you'll find a bunch of links, like Ryan said. And you can also download the PDF copy of the slides from today. And you'll always be able to come back and watch on demand. And you can use the same link that brought you here. And we'll be sending that link again shortly. So let me see.

here, so we have a good amount of questions that have come in, but I think all your colleagues have been hard at work answering them. So, you'll see some questions in the feed there, but for the most part, we've got them answered. Um, let's see. So, we just got a new one that came in. The example that you showed uses the AMD model zoo. How can we handle open source or custom models? It's a great question. Um, we actually have on our GitHub, if you look at the examples folder, there is basically an example code that shows how we use Vitis AI for inferring one of the models that we don't have that basically wrapped into one of the nodes. So, basically, it shows a custom code that you can run your own inference code. So, as you know, in most of these models, almost all the models we run right now use AMD DPU. So, that example shows how I can use just a DPU after you convert your model.

from the floating point and quantize and compile it for career some you can basically um use this that is an example of how to infer your own uh custom model um so i hope that answered your question. One point is jeff um i think i guess there was some sort of a sinking issue in the beginning and ryan was going over the image of uh basically uh the image that we have everything overview of the our vmss i think maybe it'd be good to just have that up here because um people missed that and that's not going to be part of this site so maybe i don't know if that's possible but ryan can just maybe share that screen because that was pretty intuitive way of showing how vms is structured into the data flow and all the options available and i think during that time during the same issue people were seeing a different video being played so sure yeah jeffrey is it possible for me to share my screen during this q a and we can all still do the chat.

yeah, so Maddie, you're probably better to speak to this. Um, yeah, I think you did a great job explaining people just not seeing this while you're. But basically, this is a big overview of how you can basically construct a pipeline left to right in terms of data flow. Um, you could pick your own video source and video sync or output. In between, you can basically use a combination of our logic application, post-processing, and output. So, in this tutorial, we basically covered RTSP video file and USB. We didn't do images, but that's also available if you need to do some sort of test. Those nodes in the middle, such as the box detector, box segmentary, box classifier, and those are highly optimized notes that we have that are capable of running AMD models and models by just configuring those a few parameters, such as that model kernel name, model type. And then you get to go to basically take advantage of MD models. Is the question that I was just asked requires you to create your own logic application node, which we have provided all the tools you need for that, such as, you know, the boilerplate that you can take.

advantage of as and write your own logic. I think the rest is more or less self-explanatory. You can basically hopefully get a good grasp of what vmss offers by just taking a look at the image, but feel free to ask any questions you need. You can always ask more questions through our GitHub repo on their issues. Feel free to drop questions, and one of our great engineers will be able to answer them. So, I see a few questions here. I don't know if they were answered elsewhere, but one of them is asking, Does the DPU is the DPU unit implemented on the Crea FPA for acceleration? The answer to that is yes, as Matthew mentioned. So, if you're familiar with this, the DPU is the deep learning processing unit, and this is the engine where AI inferencing runs on the FPGA, the programmable logic portion of the Crea System on Module. So, that's correct. It is using that, and it's using Vitus AI as the AI toolkit framework for that whole the compilation.

the quantization of a of a standard model from your framework, such as PyTorch, TensorFlow, and bringing it down into this to the executable format of instructions for the DPU. I also see another question asking if this supports ONNX, and yes and no, it's not a standard framework supported by this version of Vitis AI, but it is possible to take a model from in an ONNX format and just convert it and take it through the tool chain. So there's just some steps to convert your model to to run on the Crea song. And I see another question, does the V does BMSS 2.0 support KV 260, and absolutely yes it does. There's really no difference from what's running under the hood. I think the difference is primarily some IOs in the kits between the KV V is for vision and the KR for robotics. So this is more of a matter of IO differences and not about the the actual device. So yes, it does is supported on the vision version of the Crea song. I think for the next question, Ryan is, is there any transfer

learning. Do you do to recognize a face during inference? The transfer learning that I'm aware of is more of a, you know, during training a model. So, I'm not sure if I understand the question correctly, but no, in terms of the things that we showed. If you're talking about transfer learning in terms of during training, that's basically we're using AMD models in these examples, and we're not providing any opera train model here. In terms of the maximum frame rates, which is what is the maximum frame rate you can support, it really depends on your input size. It depends on what sort of pipeline you want to construct for yourself. We provide two different video decoder and encoder, one is using the hardware accelerated version, the other one the software. Also depends on that and the model size. So, I do recommend that it's free to for you to try, go ahead and try it, and hopefully that matches your need. Can you do real-time face recognition? Yes, you can, given that you construct the optimized software. So, to answer this question differently, the career song is using five EV FPJ. Do you guys, we have provided.

to another company a face recognition application on a seven ev. But we could do about four streams of real time so this should be able to support at least one to two or maybe more. Yeah, yeah. So about that frame rates and how many streams can you support it. Yeah, it really depends on the model that you're running. These the device itself is absolutely capable of supporting many streams. And when it comes to the video, the video transcoding portion. So a lot of times you're you may not be limited by the AI performance of the device. You may be limited by the video transcoding capability because you're often taking in a live video stream that's encoded with H264 and perhaps you're even outputting it to an encoded video stream. So that can be pretty expensive. This device has a VCU. So it's a video codec unit that is capable of doing eight streams of encoding, decoding, and encoding. So it has a pretty high capability in the video transcoding alone. And so then it really depends on the model you're using. I see a question of

does the kv2 60 have any board supporting three cameras as of today. That really depends on the type of camera you're using so I don't know how to answer that. If you mean like a hardware camera we have to talk some more but if you're using a network camera. So when we were showing the example of using an rtsp video stream you're not using hardware for as an input. This is coming over a network protocol so there there's no limitation there you can have eight 16 then you're just gated by your transcoding capability of the system. So I think there's one more question that I don't quite understand the stream is not important but rather the detection results so they're both follow the question mark but the detection results. I guess like I can talk about that you have every way of consuming that you can create your own custom basically post-processing node you can use one of the ones that we have available for notification as you can see on the right side of this image that we're showing.

is up to you how you want to use your results. These are what we provide in terms of the available nodes. You can always use your own custom logic. So hopefully that answers your question. The streaming could be important to some people. Yeah, I think maybe one way I interpreted that is as I was showing the output results and you know there were some macro blocking and some degraded image in the result. And so if that's if that was the question, is why does the result of the encoded video stream look so poor. That's because it takes a little bit of time of buffering. The codec needs to kind of take in a certain number of frames before it reaches a steady state. And then the results become very clear and true to the input. So perhaps that's what you were seeing. And it's just for the sake of time. I was killing that pipeline before the quality of the output stream really reached its capability. I see another question. Can you provide contact info for the pervasive AI developer contest that vmss is supporting it

it is going to be one of the links included and posted in this webinar. And I see a question about the lead time for the device. I'm sorry, I don't have an answer to that. But we can, I think, if you reach out to us, we can, we can try to get you in contact with one of the suppliers to answer that question. Okay, cool. Okay, I think that is it. Let me do one last check here. Yeah, I think that is all we've got in the queue. Uh, so yeah, I think we are ready to, uh, to wrap up here. Wrap it up, yeah. Thank great. Thank you, everybody. Thanks for, uh, for joining us and sticking around here to the end of Q&A. You can fill out that survey for us before you leave. That would help us out a lot. So thank you for that in advance. And you'll be able to come back and watch on demand whenever you want. So thank you. Thank you again. Thanks, everybody. Thank you, everyone. Good luck using VMSS. Thank you.

Maximizing Video AI Performance: Running Video Analytics on AMD Devices with VMSS2.0

Get the competitive edge for AI, data center, business computing solutions & gaming with AMD processors, graphics, FPGAs, Adaptive SOCs, & software.

The world’s most advanced processors
For the world’s most ambitious ideas

AMD processors power people to lead their fields at the cutting edge. From healthcare and entertainment to science and autonomous driving—when AMD performance meets the potential of our partners, we can solve the world’s most important challenges.

together we advance_



Maximizing Video AI Performance: Running Video Analytics on AMD Devices with VMSS2.0

Maximizing Video AI Performance: Running Video Analytics on AMD Devices with VMSS2.0

스크립트