Person Recognition

We can view the WiFi camera screen through the APP, and the car will respond accordingly based on the recognition results.

Since the previous tutorial has explained the basic knowledge of all car modules, the car control tutorial will not be repeated, and the main focus is on the implementation ideas of the car functions!

Device connection

Hardware connection

Use Type-B data cable to connect Arduino Uno and computer.

Before this, you need to unplug the connection between the WiFi camera and Roboduino, otherwise the serial port will be occupied and the computer cannot recognize the serial port number of Arduino Uno.

Software connection

Open the "Arduino IDE" software and select the model and serial port number corresponding to the development board.

Implementation ideas

Analyze the data sent by the WiFi camera through the serial port and control the car according to the data.

Code analysis

Here is only a brief introduction to the code content. For detailed code, it is recommended to refer to the corresponding code file, which is provided in the download area!

1. esp32_wifi

• Define a macro WIFI_MODE to select different WiFi modes based on preprocessor directives (#if)

​x
#if MODE_AP_STA
#define WIFI_MODE '2'
​
#elif MODE_STA
#define WIFI_MODE '1'
​
#elif MODE_AP 
#define WIFI_MODE '0'
​
#else
#define WIFI_MODE '2'
#endif

• Enumerate AI modes, including cat and dog detection, face detection, color detection, etc.

xxxxxxxxxx
typedef enum AI_mode_t
{
    Nornal_AI = 0, //不检测 No detection
    Cat_Dog_AI,   //猫狗检测 Cat and Dog Detection
    FACE_AI,      //人物检测 People Detection
    COLOR_AI ,   //颜色识别 Color recognition
    REFACE_AI,   //人脸识别 Face Recognition
    QR_AI = 5,   //二维码识别 QR code recognition
    AI_MAX      //最大值 Maximum
}AI_mode;

• SET_ESP_WIFI_MODE function sets the WiFi mode of the ESP32 camera.

xxxxxxxxxx
void SET_ESP_WIFI_MODE(void) //设置模式选择 Setting Mode Selection
{
  //选择模式STA+AP模式共存 Select mode STA+AP mode coexistence
  sprintf(send_buf,"wifi_mode:%c",WIFI_MODE);
  ESPWIFISerial.print(send_buf);
  memset(send_buf,0,sizeof(send_buf));
  
  delay(2000);//等待复位重启成功 Wait for reset to restart successfully
}

• The SET_ESP_AI_MODE function sets the AI mode of the ESP32 camera.

xxxxxxxxxx
void SET_ESP_AI_MODE(AI_mode Mode) //设置AI模式 Setting AI Mode
{
  
  sprintf(send_buf,"ai_mode:%d",Mode);
  ESPWIFISerial.print(send_buf);
  memset(send_buf,0,sizeof(send_buf));
  runmode = Mode;
​
  delay(2000);//等待复位重启成功 Wait for reset to restart successfully
}

2. esp_key

• The key_state function is used to detect short press and long press events of a button and return the corresponding state value.

  • 0 means the button is not pressed.
  • 1 means short press.
  • 2 means long press.

xxxxxxxxxx
//长按和短按
//长按:2 短按：1 不按：0
//Long press and short press
//Long press: 2 Short press: 1 No press: 0
int key_state()
{
  int touchReading = digitalRead(KEY_PIN);
  int8_t key_state = 0;
  if (touchReading != lastTouchState) {
    lastDebounceTime = millis();
  }
 
  if ((millis() - lastDebounceTime) > debounceDelay) {
    if (touchReading != touchState) {
      touchState = touchReading;
 
      if (touchState == LOW) {
        // 触摸按下 Touch Press
        touchDownTime = millis();
        touchInProgress = true;
        singleClick = false;
        longPress = false;
      } else {
        // 触摸释放 Touch Release
        if (touchInProgress) {
          if ((millis() - touchDownTime) < longPressDelay) 
          {
            // 短按判定 Short press judgment
            if (!singleClick) 
            {
              singleClick = true;
            }
          } 
          else 
          {
            // 长按判定 Long press judgment
            longPress = true;
          }
          touchInProgress = false;
        }
      }
    }
  }
 
  if (singleClick) {
    //ESPWIFISerial.println("1");
    key_state = 1;
    singleClick = false;
  }
  if (longPress) {
    //ESPWIFISerial.println("2");
    key_state = 2;
    longPress = false;
  }
 
  lastTouchState = touchReading;
​
  return key_state;
​
}

• The send_key function sends different command strings according to the value of Virtual_key and outputs them through the ESPWIFISerial.print function.

xxxxxxxxxx
//esp32 虚拟按键 Virtual buttons
uint8_t Virtual_key = 1; //1: Send KEY_MENU 2: Send KEY_PLAY 3: KEY_DOWN (3 only works for color recognition)
static void send_key(void)
{
  switch(Virtual_key)  
  {
    case 1:ESPWIFISerial.print("KEY_MENU");break; //RGB灯亮绿色 RGB light turns green
    case 2:ESPWIFISerial.print("KEY_PLAY");break; //RGB灯亮蓝色 RGB light turns blue
    case 3:ESPWIFISerial.print("KEY_DOWN");break; //RGB灯亮红色 RGB light turns red
    case 4:ESPWIFISerial.print("KEY_PLAY");break; //RGB灯亮蓝色 /RGB light turns blue
  }
}

• The key_goto_state function performs different operations depending on the state of the key (short press or long press). The key_state function is called to get the state of the key, and the operation to be performed is determined based on the state of the key and the system operation mode (runmode).

  • If you long-press KEY1 and call the key_RGB function to change the color of the LED, the value of Virtual_key will be incremented. If the color recognition mode is running and the value of Virtual_key exceeds 4, Virtual_key will be reset to 1;
  • If you short-press KEY1 and run the color recognition mode, call the send_key function to send the corresponding command.

xxxxxxxxxx
void key_goto_state(void)
{
  uint8_t key_oo = key_state();
    if(key_oo == 2)//长按 Long press
    {
      Virtual_key ++ ;//改变发送的命令 Change the command sent
      if(runmode == COLOR_AI) //只有颜色识别才发送KEY_DOWN Only send KEY_DOWN for color recognition
      {
         if(Virtual_key > 4)
        {
          Virtual_key = 1; 
         }
      }
      else //人物识别 Person Recognition
      {
         if(Virtual_key > 2)
        {
          Virtual_key = 1; 
         }
      }      
          key_RGB();      
     }
     
     else if(key_oo == 1)//短按 Short press
     {
        if(runmode == COLOR_AI || runmode==REFACE_AI )//Only color recognition and face recognition interact
        {
          send_key();//发送数值 Sending Values
        }       
     }  
}

3. bsp_RGB

• Include libraries such as Adafruit_NeoPixel.h and motor_car.hpp header files

xxxxxxxxxx
#include <stdio.h>
#include <string.h>
#include <Arduino.h>
#include <Adafruit_NeoPixel.h> 
#include "motor_car.hpp

• Enumerate RGB light status, off, red, green, blue, yellow, etc.

xxxxxxxxxx
// 枚举常见颜色 Enumerate common colors
enum ColorType {
  BLACK,
  RED,
  GREEN,
  BLUE,
  YELLOW,
  MAGENTA,
  CYAN,
  WHITE,
};

• Define RGB light control pins

xxxxxxxxxx
// 定义RGB控制引脚和数量 Define RGB control pins and quantity
#define RGB_PIN 6
#define RGB_NUM 1

• Create an instance of Adafruit_NeoPixel

xxxxxxxxxx
// 创建Adafruit_NeoPixel类的实例 Create an instance of the Adafruit_NeoPixel class
Adafruit_NeoPixel RGB = Adafruit_NeoPixel(RGB_NUM, RGB_PIN, NEO_GRB + NEO_KHZ800);

• Initialize RGB light control

xxxxxxxxxx
void RGB_init()
{
    RGB.begin();  // 初始化RGB Initialize RGB
    RGB.show();   // 刷新RGB显示 Refresh RGB display
    if(runmode == COLOR_AI || runmode==REFACE_AI )
    {
        setRGBColor(GREEN); //默认关闭 Default off
        
     }
     else
     {
        setRGBColor(BLACK); //默认关闭 Default off 
     }
    
}

• Set the color of the RGB light

xxxxxxxxxx
/**
 * @brief 设置RGB显示的颜色 Set RGB display color
 * @param color: 显示的颜色 Set the color
 * @retval 无 None
 */
void setRGBColor(ColorType color) {
  switch (color) {
    case RED:
      RGB.setPixelColor(0, RGB.Color(255, 0, 0));
      RGB.show();
      break;
    case GREEN:
      RGB.setPixelColor(0, RGB.Color(0, 255, 0));
      RGB.show();
      break;
    case BLUE:
      RGB.setPixelColor(0, RGB.Color(0, 0, 255));
      RGB.show();
      break;
    case YELLOW:
      RGB.setPixelColor(0, RGB.Color(255, 255, 0));
      RGB.show();
      break;
    case MAGENTA:
      RGB.setPixelColor(0, RGB.Color(255, 0, 255));
      RGB.show();
      break;
    case CYAN:
      RGB.setPixelColor(0, RGB.Color(0, 255, 255));
      RGB.show();
      break;
    case WHITE:
      RGB.setPixelColor(0, RGB.Color(255, 255, 255));
      RGB.show();
      break;
    default:
      RGB.setPixelColor(0, RGB.Color(0, 0, 0));
      RGB.show();
      break;
  }
}

4. face_study_esp32.ino

• Import header files, including the implementation of ESP32's WiFi function, motor control, button management, and RGB light control.

xxxxxxxxxx
#include <stdio.h>
#include "esp32_wifi.hpp"
#include "motor_car.hpp"
#include "esp_key.hpp"
#include "bsp_RGB.hpp"

• Here a macro AI_set_mode is defined, indicating that the current AI mode is REFACE_AI (person recognition).

cmd_flag is an external variable used to indicate the current command status.

xxxxxxxxxx
#define AI_set_mode REFACE_AI  //设置AI模式 Setting AI Mode
#define BUZZER_PIN 10
​
extern uint8_t cmd_flag;

• mode_chage() is a mode switching function that sets the value of cmd_flag according to the current operating mode to indicate different processing modes.

xxxxxxxxxx
void mode_chage()
{
  if(runmode == Nornal_AI) 
  {
    cmd_flag = 2;//进入透传模式 Enter transparent mode
  }
  else if(runmode == REFACE_AI )
  {
    cmd_flag = 3;//解析人脸识别数据 Parsing facial recognition data  
  }
  else if(runmode == QR_AI )
  {
    cmd_flag = 4;//解析二维码数据  Parsing QR code data
  }
  else
  {
    cmd_flag = 5;//其它AI模式数据  Other AI mode data
   }
}

• When the decect_study_face() function is used for person detection, different responses are made according to the detection results.

  • When an unfamiliar face is detected (not learned before), the car's RGB light turns red and the horn sounds for 1 second;
  • When a known face (one that has been learned) is detected, the car's RGB light turns green.

xxxxxxxxxx
void decect_study_face()
{
   if(esp32_ai_msg.id <= 0)//不认识的人脸 Unrecognizable face
   {
      setRGBColor(RED); //亮红灯 Red light
      tone(BUZZER_PIN, 1000);  //鸣笛 play
      delay(100);
      noTone(BUZZER_PIN);  // 停止发声 Stop play
      delay(300);
      setRGBColor(BLACK);
   }
   else //认识的人脸 Recognizable faces
   {
      setRGBColor(GREEN);  //亮绿灯 Green light
      delay(500);
      setRGBColor(BLACK); //关闭 close
   }
}

• The setup() function is the entry point for initializing the program, including the initialization of the buzzer, button, serial port, WiFi, AI mode, motor, and RGB light, and calls the mode_chage function to set the command flag and enter the face entry process.

  Face entry process:

  • Use a loop to wait for the user to press a key;
  • If the user briefly presses the button, a command is sent to start face recording;
  • If the user long presses the button, the loop will be exited and the face entry process will end;
  • Use the buzzer to sound a reminder.

xxxxxxxxxx
void setup()
{
  pinMode(BUZZER_PIN, OUTPUT); //蜂鸣器初始化 Buzzer initialization
  init_key();//按键初始化 Button initialization
  serial_init();//串口初始化 Serial port initialization
  SET_ESP_WIFI_MODE();//设置wifi模式 Set Wi-Fi mode
​
  SET_ESP_AI_MODE(AI_set_mode);//设置AI模式 Setting AI Mode
​
  Motor_init();//电机初始化 Motor initialization
  RGB_init();//RGB初始化 RGB initialization
​
  mode_chage();//根据模式选择解析数据办法 Choose the method to parse data according to the mode
​
  //Start recording the face first, and then directly perform face recognition after recording. Long press the key
  uint8_t key_num = 0;
  do
  {
    key_num = key_state();
    if(key_num == 1)
    {
      ESPWIFISerial.print("KEY_MENU"); //录取人脸 Admission face
      key_num = 0;//清空一下 Clear it
    }   
    
   }while(key_num !=2 );
​
​
   tone(BUZZER_PIN, 1000);  //鸣笛 play
    delay(100);
    noTone(BUZZER_PIN);  // 停止发声 Stop play
    delay(300);
​
​
}

• Define a variable previousMillis of unsigned long type to store the timestamp (in milliseconds) of the last time an operation was performed.

xxxxxxxxxx
unsigned long previousMillis; //ms

• In the main loop, if the time from the last command to now exceeds 1500 milliseconds (1.5 seconds), send the "KEY_PLAY" command and update previousMillis. If the newlines variable is 1, it means that new data has been received. Set newlines back to 0, indicating that the new data has been processed, and then call the decect_study_face function to process the received face data.

xxxxxxxxxx
void loop() 
{
  unsigned long currentMillis = millis();
  if(currentMillis-previousMillis > 1500)//Send once every 1s
  {
      ESPWIFISerial.print("KEY_PLAY"); 
      previousMillis = currentMillis;
   }
   
  
  if(newlines == 1)//接收到新数据 New data received
  {
     newlines = 0;
//     ESPWIFISerial.println("aa");
//     ESPWIFISerial.println(esp32_ai_msg.id);
​
     //识别到人脸 Recognize the face
    decect_study_face();
  }
  
}

Experimental results

After compiling the program successfully, upload the code to the Arduino Uno development board, disconnect the car from the computer, and connect the WiFi camera to the serial port on the expansion board.

After the program starts, we can observe the camera screen through the YahboomCam APP.

APP connection

Connect the mobile phone to the hotspot of the WiFi camera (the name of the built-in hotspot: Yahboom_ESP32_WIFI), and then open the YahboomCam software.

xxxxxxxxxx
Some mobile phones will prompt for connecting to a hotspot without a network, and we need to click to keep connected!

Enter IP:192.168.4.1 in the YahboomCam software, then click Login to enter the APP control interface.

xxxxxxxxxx
The IP of the WiFi camera's built-in hotspot is 192.168.4.1

After entering the APP interface, the APP will display the camera screen.

xxxxxxxxxx
If the screen is not displayed, check whether the phone is connected to the WiFi camera hotspot normally

Effect

Aim the camera at the face to be learned

Press KEY1 to confirm the face to be enrolled. After each confirmation, a prompt will appear on the screen

If you don't want to record the face, press and hold the KEY1 key, then release it. When you hear a beep, the face recognition is ready.

When an unknown face is recognized, the car RGB lights up red and whistles, and the screen is displayed as follows

When a known face is recognized, the car RGB lights up green, and the screen is displayed as follows

Face:

Notes

1. The screen needs to be turned on to start recognition. If the screen is turned off, the recognition will also be turned off.
2. Make sure the direction of the camera image is correct and turn on the vertical rotation and horizontal rotation switches on the APP.