[Music] Hey people, in this video, I will show you how you can use machine learning for object recognition with the help of an ESP32 development board and Edge Impulse. Edge Impulse is an online platform using which you can easily build and deploy machine learning models very easily. We have already built a lot of projects with it, and the best part is it's completely free to build small projects for individuals. For this project, I'll show you how you can train this ESP32 CAM module to detect potatoes, tomatoes, and onions, but you can train it to detect anything of your choice. For more such useful tutorials and projects, consider subscribing to our channel. This is Ashwin. You're watching Circuit Digest. Let's get [Music] started. First, let's start with the hardware connections required for this project. Okay, so this is the complete hardware setup of our project. We just have an ESP32 CAM module on a breadboard connected to an OLED screen and an FTDI programmer module. You can use any USB to TTL programming module. So, this programming module is connected to the

VCC and ground to power our ESP32 cam board, and we also have a jumper wire over here for GPIO0, which we need to connect to ground during programming and remove it during operation. And this OLED screen will display which object is being detected. So, this is the complete circuit diagram for our project. This is used to program our ESP32, and this is used to display our result. If you're confused on what these pin names are, you can use these pin-out images for reference. So now that we know that the hardware is really simple, let's take this to our computer and show how to program this for object identification. Okay, now we are on our computer with the hardware in my hand. I have the OLED display disconnected. We will connect it after we are done with the complete project. So what I'm going to do is I'm going to connect this to my laptop over here using this programming cable. Once it's connected, we'll open Arduino IDE. And before we do all that, if you're a complete newbie with ESP32 Cam, then you

should definitely consider checking out this article on our website called How to Program ESP32-CAM. So, this article will tell you how to upload code, how to prepare your Arduino IDE, how to install the right board managers, and everything. So, if you're a complete newbie, please do check this out and follow this tutorial to know how to upload code on your ESP32. Now, one more thing I should say is that I'm going to use this ESP32-CAM board itself to collect all the images. In fact, let's quickly get into Edge Impulse, and I'm going to log in because I have already created an account. But if you haven't, you can register. It's completely free. Once you log in, you can create a new project. I will call this Veg Detect, and I will create a new project. Okay, as you can see, the first step we have to do in Edge Impulse is collect new data. So, in our case, we have to collect data of potatoes, tomatoes, and onion. So, all these images you can either take directly from websites like Kel

they have huge data sets. Or you can even download from Google. But what we're going to do is we're going to use this ESP32 cam setup itself to take images of all these three things and use those images to train our model. This is better because this ESP32 cam doesn't have a lot of processing power. So if you take images using the same camera and if you use the same camera to detect your objects, you will get better results. So whatever object recognition project you're doing, first create the complete hardware setup. And let's say you're reading something from the conveyor, install the camera on the conveyor, finish your hardware setup first, then take test images in the same way that your actual demo would also look like, and then use those test images to train your data. So that's what we're going to do. And to do that, we are going to place our ESP32 cam in some position. And this is the position that we will remain, that it will remain forever in this project. So I'm going to mark this.

position using a marker, so that you know just in case if it gets accidentally moved. I can place it in the same position. So now that we have our ESP32 cam in position, let's go back to Arduino IDE, and we'll install a library to have a code, which will read images from our ESP32 Cam, and it will store those images on our computer. So to do that, go into sketch, include Library, manage library, and here you have to search for ELO Quint ESP32. So this ELO Quint ESP32 cam, you can install it, but I have already installed. So since that is done, I'll close this and go to file examples. And all the way down, you'll find eloquent tp3 to collect images from H impulse. So open this example code, and once it's opened, you have to make some changes onto it. So the first thing would be the SSID, which in my case is semicon media, and the password is something I won't tell you. And all the way down, you have to change this to AI.

think once that is done, you can go into tools and make sure that the board is selected as AI thinker. The right port is selected, and before you hit on upload, you also have to make sure that your GPIO Z is pulled to high. That's what is explained in the article, which I told you earlier. Do a power reset and click on upload. Okay, so now the code is uploaded. You can remove this jumper wire over here and do another power reset and open the serial monitor. You will see that the Wi-Fi is connected, and it will also give you an image collection server link. We just copy this and head over to Chrome and open this link. Make sure you're connected to the same network to which you connected your ESP32 as well. Now, over here, you can see that we are getting the image from our camera. So, this is what we're going to use to train our onion, tomato, and potato. So, first, let's place the onion in the frame. So now, to collect images, what you have to do is just click on start.

collecting it will start taking pictures, and then you can rotate the object in different orientations. Click on start collecting again, stop it again, rotate it again, start collecting again, stop it again. So you can collect up to 50 images of whatever object you want to recognize. Once you're done with that, you can click on download and give a label. So I'm going to just call it onion. It will download it as a zip file. All the images taken here are downloaded as a zip file. We will then be using this onage impulse. So I'm going to keep this. And then once you're done with one object, you can place the other object in the same circle. You can see that we have placed the tomato now, and then we can rotate this tomato also in different orientations. So we'll clear this, start collecting, stop, rotate, start collecting, stop, rotate, start collecting, stop. So you can do this for all the objects that you want to do. I have already done it and saved all the images, which I will be needing. And you can see all the images.

over here, these are the images of my tomato, images of onion, and images of potato. So now we have all the images required to train our model. Let's get into Edge Impulse and start training. Okay, now what we do is we get back to our Edge Impulse platform. And then the first thing we want to do is data acquisition. So click on data acquisition and then click on add data and then upload data. Now, there are multiple ways to add data. You can also use a QR code method. You can also directly take pictures from your phone and add data. But what I showed you gives the best results. Now, to upload data, you can select a folder. And then under choose files, you can go to the place where we have collected all the images. First, we'll start with images of onion. Select the complete file and click on upload. Yes, only if you trust this. Yes, we can click on upload. And then automatically split. And then we can manually enter and label. Since we have selected onion, we will select onion. You can see it in the suggestion since I

have already tried this project, and then click on upload project. So, once the data is uploaded, it will ask if you want to change the labeling method. Just click on yes, and all the data is uploaded. So, you can do this three times to upload for all the various vegetables. So, I'm going to do that now quickly. Okay, so now I have uploaded for all the three vegetables. So, now you can see that we have uploaded a total of 56 items, and labeling queue is pending for all 56. So, let's give a label for all the 56 images which we have uploaded. So, click on labeling queue, and here we can go to track objects between frame, and you have to select where the onion is. So, we just drag a frame like this, and we will call it onion. So, the label is set. You can see this is the onion. Click on Save label, and then it will give you the next object. You have to make sure the frame is positioned correctly for the next object as well, and then click on Save label. So, similarly, you have to do this.

for all the images. Okay, now we get a potato. So what we will do is we will change this label to potato, and you can see that the color is also changed, and we will adjust the frame to train it for potato. Now let's do the same for tomato as well. Okay, so now you can see that the labeling queue has become zero. We have labeled all the 56 images. The next step is to design an Impulse. To do that, click on create impulse, and then we're going to leave the image as 96 cross 96, fit shortest axis. We leave all these to the default, and then under processing block, we will add the image processing block from Edge impulse, and then learning block will be object detection. Once that is done, click on Save impulse. You can see that the output features are three: onion, potato, and tomato. So this comes from the label which we have created earlier. The next one is you can get into generating features. So to do that, click on image, and then color depth, we're going to

change it to grayscale because it's much easier to work with grayscale images than color images for a low-cost development board like esp32. So we're going to select grayscale and save parameters. Then click on generate features. It will take some time, and once it's done, you can see this kind of a map, which shows how each of your objects is different from the other object. You want all these dots to be clustered as far from one another as possible. You can see that the onions here are far away from the tomatoes, and the potatoes here have slightly mixed with the tomato. So your system is going to have some difficulty distinguishing between a tomato and potato as you will see in the end result. But yeah, this map gives you a clear idea on how different each of your objects is from the other. The farther they are from the other clusters, the better it will work. Now, since this is done, we'll get into object detection. And then here, the number of training cycles, you can keep at 60.

this is where we going to train all our data so we can keep the training cycles as 60. So it will train 60 times up and then the lower the learning rate is, the faster your model will get trained. 0.001 is a good learning rate, but it take a lot of time. So I'm going to do it as 0.01. And then training processor will be CPU. With all of this selected, I'm going to click on start training. This is going to take some time, maybe a couple of minutes to up to 10 minutes based on your number of images and the number of training cycles and learning rate you have put. So let's wait for this to get done. Okay, so now our model has finished training. And this is what you will see. So this means we have trained our first machine learning model. And you can see the results here. It is able to identify onion with 100% accuracy, and potato and tomato all of that is able to be identified with 100% accuracy. This is a

very good score. You might get anywhere between 90 to 100. All of that is practically a good score. Now, once we have done training, all we have to do is deploy it. You can get into deployment, and here you can select already know library because we're going to do it for the ESP32 CAM module. So, this is the module we're going to use. So, then you can come all the way down and then change Target. You can search for ESP. You won't find the ESP32 CAM. So, we're going to select the ESPi. So, let me select that, select Target device, and then build. Okay, now you can see that we have successfully built our code, and it is automatically downloaded as EIW dedu ordino like a library package. We have to add this to our ordino code. So, it's called V dedu ordino. So, we'll go back to our Ordo, and then we'll do sketch, include library, add a zip file. Want to go into downloads, and based on date modified, you will see that EIdetect orino is the

code which we generated using Edge impulse so we will choose this. Now if you get into files examples, you will see that vegge dedu interfacing is available here. We'll select that and it's for ESP32 camera so we will select that board. This is the code for our object deduction so we'll just make this bigger. If you scroll down, you will see that it's by default written for ESPi so we will remove that and enable this to make it work for ESP32 cam module. With that corrections done, we can directly upload this again on the hardware side. Before you upload, make sure that your GPIO zero is connected to ground and you do a power recycle just like we did last time. Okay, now after almost 5 minutes, the code finally got compiled and uploaded. So what we're going to do is we're going to remove this jumper cable from here and do a power reset again and place it on the exact same position. So now we can open our serial monitor.

and you can see that if I place the box right, you can see that we are able to deduct the objects. So first let's place a potato. You can see the potato is being discovered with almost 53% accuracy. Now let's remove the potato and place an onion in the circular frame. And you can see that the onion is being deducted with 99% accuracy. So yeah, you can place various objects into the circle now, and our ESP32 module will detect it and print it on the serial monitor. Okay, so now what I'm going to do is I'm going to connect this OLED display to our ESP32 CAM module over here and modify this code a little bit to show the result on this OLED display and show you how everything is working. So let me do that. [Music] With that, we have come to the conclusion of this video. Hope you enjoyed watching it and learn something useful. If yes, do consider liking this video. And if you have any problems in getting this project to work, leave them in the

comment section, and I will try my best in answering them. Have a good day. Bye-bye.

How to do Object Detection using ESP32-CAM and Edge Impulse YOLO Model

How to do Object Detection using ESP32-CAM and Edge Impulse YOLO Model