ESP32를 사용한 객체 인식: 초보자를 위한 머신 러닝

Ashwin이 ESP32와 Edge Impulse를 사용하여 감자, 토마토, 양파를 감지하도록 머신러닝을 학습시켜 물체 인식에 머신러닝을 사용하는 방법을 시연합니다.

### ESP32 및 엣지 임펄스를 사용한 객체 인식
* 🔌 하드웨어 설정에는 **ESP32 CAM 모듈, OLED 스크린, 브레드보드에 연결된 FTDI 프로그래머 모듈**이 포함됩니다.
* 📸 감자, 토마토, 양파의 이미지는 **ESP32 CAM 모듈을 사용**하여 수집하고 모델 학습을 위해 저장합니다.
* 🧠 수집된 이미지는 데이터 수집 및 라벨링을 위해 **엣지 임펄스에 업로드**됩니다.
* 🎨 이 모델은 저비용 하드웨어에서 더 나은 성능을 위해 이미지 처리를 **회색조로 설정한 Edge Impulse**로 설계되었습니다.
* 🔄 이 모델은 CPU에서 **60주기와 0.01의 학습률**로 학습됩니다.
* ✅ 학습된 모델은 **감자, 토마토, 양파를 100% 정확하게 감지**합니다.
* 📊 모델은 **ESP32 CAM 모듈에 배포**되고 결과는 **직렬 모니터에 표시**됩니다.
* 🖥 ️ 코드를 수정하여 **OLED 화면에 결과를 표시**하여 물체 인식 프로젝트를 완료합니다.

[Music] Hey people, in this video, I will show you how you can use machine learning for object recognition with the help of an ESP32 development board and Edge Impulse. Edge Impulse is an online platform using which you can easily build and deploy machine learning models very easily. We have already built a lot of projects with it, and the best part is it's completely free to build small projects for individuals. For this project, I'll show you how you can train this ESP32 CAM module to detect potatoes, tomatoes, and onions, but you can train it to detect anything of your choice. For more such useful tutorials and projects, consider subscribing to our channel. This is Ashwin. You're watching Circuit Digest. Let's get [Music] started. First, let's start with the hardware connections required for this project. Okay, so this is the complete hardware setup of our project. We just have an ESP32 CAM module on a breadboard connected to an OLED screen and an FTDI programmer module. You can use any USB to TTL programming module. So, this programming module is connected to the

VCC and ground to power our ESP32 cam board, and we also have a jumper wire over here for GPIO0, which we need to connect to ground during programming and remove it during operation. And this OLED screen will display which object is being detected. So, this is the complete circuit diagram for our project. This is used to program our ESP32, and this is used to display our result. If you're confused on what these pin names are, you can use these pin-out images for reference. So now that we know that the hardware is really simple, let's take this to our computer and show how to program this for object identification. Okay, now we are on our computer with the hardware in my hand. I have the OLED display disconnected. We will connect it after we are done with the complete project. So what I'm going to do is I'm going to connect this to my laptop over here using this programming cable. Once it's connected, we'll open Arduino IDE. And before we do all that, if you're a complete newbie with ESP32 Cam, then you

should definitely consider checking out this article on our website called How to Program ESP32-CAM. So, this article will tell you how to upload code, how to prepare your Arduino IDE, how to install the right board managers, and everything. So, if you're a complete newbie, please do check this out and follow this tutorial to know how to upload code on your ESP32. Now, one more thing I should say is that I'm going to use this ESP32-CAM board itself to collect all the images. In fact, let's quickly get into Edge Impulse, and I'm going to log in because I have already created an account. But if you haven't, you can register. It's completely free. Once you log in, you can create a new project. I will call this Veg Detect, and I will create a new project. Okay, as you can see, the first step we have to do in Edge Impulse is collect new data. So, in our case, we have to collect data of potatoes, tomatoes, and onion. So, all these images you can either take directly from websites like Kel

they have huge data sets. Or you can even download from Google. But what we're going to do is we're going to use this ESP32 cam setup itself to take images of all these three things and use those images to train our model. This is better because this ESP32 cam doesn't have a lot of processing power. So if you take images using the same camera and if you use the same camera to detect your objects, you will get better results. So whatever object recognition project you're doing, first create the complete hardware setup. And let's say you're reading something from the conveyor, install the camera on the conveyor, finish your hardware setup first, then take test images in the same way that your actual demo would also look like, and then use those test images to train your data. So that's what we're going to do. And to do that, we are going to place our ESP32 cam in some position. And this is the position that we will remain, that it will remain forever in this project. So I'm going to mark this.

position using a marker, so that you know just in case if it gets accidentally moved. I can place it in the same position. So now that we have our ESP32 cam in position, let's go back to Arduino IDE, and we'll install a library to have a code, which will read images from our ESP32 Cam, and it will store those images on our computer. So to do that, go into sketch, include Library, manage library, and here you have to search for ELO Quint ESP32. So this ELO Quint ESP32 cam, you can install it, but I have already installed. So since that is done, I'll close this and go to file examples. And all the way down, you'll find eloquent tp3 to collect images from H impulse. So open this example code, and once it's opened, you have to make some changes onto it. So the first thing would be the SSID, which in my case is semicon media, and the password is something I won't tell you. And all the way down, you have to change this to AI.

think once that is done, you can go into tools and make sure that the board is selected as AI thinker. The right port is selected, and before you hit on upload, you also have to make sure that your GPIO Z is pulled to high. That's what is explained in the article, which I told you earlier. Do a power reset and click on upload. Okay, so now the code is uploaded. You can remove this jumper wire over here and do another power reset and open the serial monitor. You will see that the Wi-Fi is connected, and it will also give you an image collection server link. We just copy this and head over to Chrome and open this link. Make sure you're connected to the same network to which you connected your ESP32 as well. Now, over here, you can see that we are getting the image from our camera. So, this is what we're going to use to train our onion, tomato, and potato. So, first, let's place the onion in the frame. So now, to collect images, what you have to do is just click on start.

collecting it will start taking pictures, and then you can rotate the object in different orientations. Click on start collecting again, stop it again, rotate it again, start collecting again, stop it again. So you can collect up to 50 images of whatever object you want to recognize. Once you're done with that, you can click on download and give a label. So I'm going to just call it onion. It will download it as a zip file. All the images taken here are downloaded as a zip file. We will then be using this onage impulse. So I'm going to keep this. And then once you're done with one object, you can place the other object in the same circle. You can see that we have placed the tomato now, and then we can rotate this tomato also in different orientations. So we'll clear this, start collecting, stop, rotate, start collecting, stop, rotate, start collecting, stop. So you can do this for all the objects that you want to do. I have already done it and saved all the images, which I will be needing. And you can see all the images.

over here, these are the images of my tomato, images of onion, and images of potato. So now we have all the images required to train our model. Let's get into Edge Impulse and start training. Okay, now what we do is we get back to our Edge Impulse platform. And then the first thing we want to do is data acquisition. So click on data acquisition and then click on add data and then upload data. Now, there are multiple ways to add data. You can also use a QR code method. You can also directly take pictures from your phone and add data. But what I showed you gives the best results. Now, to upload data, you can select a folder. And then under choose files, you can go to the place where we have collected all the images. First, we'll start with images of onion. Select the complete file and click on upload. Yes, only if you trust this. Yes, we can click on upload. And then automatically split. And then we can manually enter and label. Since we have selected onion, we will select onion. You can see it in the suggestion since I

have already tried this project, and then click on upload project. So, once the data is uploaded, it will ask if you want to change the labeling method. Just click on yes, and all the data is uploaded. So, you can do this three times to upload for all the various vegetables. So, I'm going to do that now quickly. Okay, so now I have uploaded for all the three vegetables. So, now you can see that we have uploaded a total of 56 items, and labeling queue is pending for all 56. So, let's give a label for all the 56 images which we have uploaded. So, click on labeling queue, and here we can go to track objects between frame, and you have to select where the onion is. So, we just drag a frame like this, and we will call it onion. So, the label is set. You can see this is the onion. Click on Save label, and then it will give you the next object. You have to make sure the frame is positioned correctly for the next object as well, and then click on Save label. So, similarly, you have to do this.

for all the images. Okay, now we get a potato. So what we will do is we will change this label to potato, and you can see that the color is also changed, and we will adjust the frame to train it for potato. Now let's do the same for tomato as well. Okay, so now you can see that the labeling queue has become zero. We have labeled all the 56 images. The next step is to design an Impulse. To do that, click on create impulse, and then we're going to leave the image as 96 cross 96, fit shortest axis. We leave all these to the default, and then under processing block, we will add the image processing block from Edge impulse, and then learning block will be object detection. Once that is done, click on Save impulse. You can see that the output features are three: onion, potato, and tomato. So this comes from the label which we have created earlier. The next one is you can get into generating features. So to do that, click on image, and then color depth, we're going to

change it to grayscale because it's much easier to work with grayscale images than color images for a low-cost development board like esp32. So we're going to select grayscale and save parameters. Then click on generate features. It will take some time, and once it's done, you can see this kind of a map, which shows how each of your objects is different from the other object. You want all these dots to be clustered as far from one another as possible. You can see that the onions here are far away from the tomatoes, and the potatoes here have slightly mixed with the tomato. So your system is going to have some difficulty distinguishing between a tomato and potato as you will see in the end result. But yeah, this map gives you a clear idea on how different each of your objects is from the other. The farther they are from the other clusters, the better it will work. Now, since this is done, we'll get into object detection. And then here, the number of training cycles, you can keep at 60.

this is where we going to train all our data so we can keep the training cycles as 60. So it will train 60 times up and then the lower the learning rate is, the faster your model will get trained. 0.001 is a good learning rate, but it take a lot of time. So I'm going to do it as 0.01. And then training processor will be CPU. With all of this selected, I'm going to click on start training. This is going to take some time, maybe a couple of minutes to up to 10 minutes based on your number of images and the number of training cycles and learning rate you have put. So let's wait for this to get done. Okay, so now our model has finished training. And this is what you will see. So this means we have trained our first machine learning model. And you can see the results here. It is able to identify onion with 100% accuracy, and potato and tomato all of that is able to be identified with 100% accuracy. This is a

very good score. You might get anywhere between 90 to 100. All of that is practically a good score. Now, once we have done training, all we have to do is deploy it. You can get into deployment, and here you can select already know library because we're going to do it for the ESP32 CAM module. So, this is the module we're going to use. So, then you can come all the way down and then change Target. You can search for ESP. You won't find the ESP32 CAM. So, we're going to select the ESPi. So, let me select that, select Target device, and then build. Okay, now you can see that we have successfully built our code, and it is automatically downloaded as EIW dedu ordino like a library package. We have to add this to our ordino code. So, it's called V dedu ordino. So, we'll go back to our Ordo, and then we'll do sketch, include library, add a zip file. Want to go into downloads, and based on date modified, you will see that EIdetect orino is the

code which we generated using Edge impulse so we will choose this. Now if you get into files examples, you will see that vegge dedu interfacing is available here. We'll select that and it's for ESP32 camera so we will select that board. This is the code for our object deduction so we'll just make this bigger. If you scroll down, you will see that it's by default written for ESPi so we will remove that and enable this to make it work for ESP32 cam module. With that corrections done, we can directly upload this again on the hardware side. Before you upload, make sure that your GPIO zero is connected to ground and you do a power recycle just like we did last time. Okay, now after almost 5 minutes, the code finally got compiled and uploaded. So what we're going to do is we're going to remove this jumper cable from here and do a power reset again and place it on the exact same position. So now we can open our serial monitor.

and you can see that if I place the box right, you can see that we are able to deduct the objects. So first let's place a potato. You can see the potato is being discovered with almost 53% accuracy. Now let's remove the potato and place an onion in the circular frame. And you can see that the onion is being deducted with 99% accuracy. So yeah, you can place various objects into the circle now, and our ESP32 module will detect it and print it on the serial monitor. Okay, so now what I'm going to do is I'm going to connect this OLED display to our ESP32 CAM module over here and modify this code a little bit to show the result on this OLED display and show you how everything is working. So let me do that. [Music] With that, we have come to the conclusion of this video. Hope you enjoyed watching it and learn something useful. If yes, do consider liking this video. And if you have any problems in getting this project to work, leave them in the

comment section, and I will try my best in answering them. Have a good day. Bye-bye.

How to do Object Detection using ESP32-CAM and Edge Impulse YOLO Model

Circuit Digest

ESP32로 객체 인식을 위한 머신 러닝 사용

엣지 임펄스는 머신러닝 모델을 무료로 구축하고 배포할 수 있는 온라인 플랫폼입니다.

이 프로젝트는 감자, 토마토, 양파를 감지하도록 ESP32 CAM 모듈을 훈련하는 방법을 보여줍니다.

하드웨어 설정에는 ESP32 CAM 모듈을 OLED 화면과 FTDI 프로그래머 모듈에 연결하는 것이 포함됩니다.

이미지 수집을 위한 ESP32 캠 사용

ESP32 캠은 물체 인식 모델 학습을 위한 이미지 촬영에 사용되며, 제한된 처리 성능을 활용하여 더 나은 결과를 얻을 수 있습니다.

카메라는 이미지 수집의 일관성과 안정성을 보장하기 위해 고정된 위치에 설치됩니다.

ELO Quint ESP32 라이브러리와 예제 코드를 사용하여 카메라는 Wi-Fi에 연결하고 양파, 토마토, 감자의 이미지를 캡처하기 위한 이미지 수집 서버 링크를 제공합니다.

엣지 임펄스에서 모델 학습을 위한 이미지 데이터 수집 및 라벨링 프로세스

이 프로세스에는 양파, 토마토, 감자 등 각 물체의 이미지를 다양한 방향으로 최대 50장까지 촬영한 다음 zip 파일로 다운로드하는 과정이 포함됩니다.

그런 다음 데이터 수집을 위해 이러한 이미지를 Edge Impulse 플랫폼에 업로드하고 프레임을 사용하여 각 물체에 대한 레이블을 식별하고 저장하기 위해 레이블을 지정하고 정리합니다.

이 라벨링 프로세스는 다양한 채소의 모든 이미지에 대해 반복되어 모델의 정확한 학습을 보장합니다.

채소 감지를 위한 머신러닝 모델 훈련하기

이 프로세스에는 감자, 토마토, 양파에 대한 이미지 라벨을 지정한 다음 물체 감지를 위한 임펄스를 설계하는 작업이 포함됩니다.

저비용 개발 보드에서 쉽게 처리할 수 있도록 회색조 이미지를 선택합니다.

모델 훈련에는 채소를 최대한 정확하게 식별하는 것을 목표로 훈련 주기 수, 학습 속도, 훈련 프로세서를 설정하는 작업이 포함됩니다.

엣지 임펄스와 함께 ESP32 캠 모듈 배포하기

교육이 완료되었으니 이제 ESP32 캠 모듈용 코드를 배포할 차례입니다.

에지 임펄스 코드가 성공적으로 빌드되어 라이브러리 패키지로 다운로드되었습니다.

Arduino IDE에 코드를 추가한 후 ESP32 캠 모듈에 필요한 수정을 하고 하드웨어에 업로드합니다.

ESP32를 사용한 객체 감지: 데모 시연

다양한 물체를 높은 정확도로 감지하는 ESP32 모듈을 선보입니다.

이 모듈은 처음에 감자를 53%의 정확도로, 양파를 99%의 정확도로 감지합니다.

데모는 결과를 표시하기 위해 ESP32 캠 모듈에 OLED 디스플레이를 연결할 계획으로 마무리됩니다.