Welcome to DOFBOT SE repository

Color Recognition

This section is mainly to prepare for subsequent feature implementations. Main steps:

• Convert the RGB image to be detected into an HSV image

• Define an object of type Mat: mask

• Define upper and lower color limits

  The upper limit is a Scalar object containing three values: hmin, smin, vmin, representing the minimum values of the three HSV elements;
  The lower limit is also a Scalar object containing three values: hmax, smax, vmax, representing the maximum values of the three HSV elements.
  

• Use the inRange function to check if each pixel of the src image is between lowerb and upperb

  xxxxxxxxxx
  If yes, set that pixel to 255 and save it in the mask image; otherwise, set it to 0.
  

Basic Principles

Common color models in digital image processing are RGB (Red, Green, Blue) and HSV (Hue, Saturation, Brightness). RGB is widely used in color monitors and color video cameras. Our usual images are generally in RGB model. The HSV model is more aligned with how humans describe and interpret colors. The HSV color description is natural and intuitive for humans. Another reason for choosing the HSV model is that RGB channels do not well reflect the specific color information of objects. Compared to RGB space, HSV space can more intuitively express the brightness, hue, and saturation of colors, facilitating color comparison.

HSV Model

HSV (Hue, Saturation, Value) is a color space created by A. R. Smith in 1978 based on the intuitive characteristics of colors, also known as the Hexcone Model. The parameters of this model are: Hue (H), Saturation (S), and Value (V).

H: 0 — 180

S: 0 — 255

V: 0 — 255

• HSV Parameter Table:

Main Code

Code path: /home/yahboom/Dofbot/6.AI_Visual/5.Color recognition.ipynb

The following code content needs to be executed step by step according to actual needs. Do not run all at once. Running the last cell will directly exit the thread.

xxxxxxxxxx
#bgr8 to jpeg format
import enum
import cv2
def bgr8_to_jpeg(value, quality=75):
    return bytes(cv2.imencode('.jpg', value)[1])

​x
#Camera component display
import traitlets
import ipywidgets.widgets as widgets
import time
# Threading library
import threading
import inspect
import ctypes
​
origin_widget = widgets.Image(format='jpeg', width=320, height=240)
mask_widget = widgets.Image(format='jpeg',width=320, height=240)
result_widget = widgets.Image(format='jpeg',width=320, height=240)
​
# Create a horizontal box container to place image widgets side by side
image_container = widgets.HBox([origin_widget, mask_widget, result_widget])
# image_container = widgets.Image(format='jpeg', width=600, height=500)
display(image_container)

Get HSV values of colors

xxxxxxxxxx
def get_color(img):
    H = []
    color_name={}
    img = cv2.resize(img, (640, 480), )
    # Convert color image to HSV
    HSV = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
    # Draw rectangle
    cv2.rectangle(img, (280, 180), (360, 260), (0, 255, 0), 2)
    # Extract H, S, V values row by row and column by column into container
    for i in range(280, 360):
        for j in range(180, 260): H.append(HSV[j, i][0])
    # Calculate min and max of H, S, V
    H_min = min(H);H_max = max(H)
#     print(H_min,H_max)
    # Determine color
    if H_min >= 0 and H_max <= 10 or H_min >= 156 and H_max <= 180: color_name['name'] = 'red'
    elif H_min >= 26 and H_max <= 34: color_name['name'] = 'yellow'
    elif H_min >= 35 and H_max <= 78: color_name['name'] = 'green'
    elif H_min >= 100 and H_max <= 124: color_name['name'] = 'blue'
    return img, color_name

Main process: Recognize red, green, blue, and yellow colors.

xxxxxxxxxx
import cv2
import numpy as np
import ipywidgets.widgets as widgets
​
​
cap = cv2.VideoCapture(0)
cap.set(3, 640)
cap.set(4, 480)
cap.set(5, 30)  #Set frame rate
cap.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter.fourcc('M', 'J', 'P', 'G'))
​
# Default to red, program will automatically switch colors based on detected color in the box
# Red range
color_lower = np.array([0, 43, 46])
color_upper = np.array([10, 255, 255])
​
​
def Color_Recongnize():
    
    while(1):
        # get a frame and show Get video frame and convert to HSV format, use cvtColor() to convert BGR to HSV, parameter is cv2.COLOR_BGR2HSV.
        ret, frame = cap.read()
        frame, color_name = get_color(frame)
        if len(color_name)==1:
            global color_lower
            global color_upper
​
            if color_name['name'] == 'yellow':
                color_lower = np.array([26, 43, 46])
                color_upper = np.array([34, 255, 255])
                
            elif color_name['name'] == 'red':
                color_lower = np.array([0, 43, 46])
                color_upper = np.array([10, 255, 255])
               
            elif  color_name['name'] == 'green':
                color_lower = np.array([35, 43, 46])
                color_upper = np.array([77, 255, 255])
              
            elif color_name['name'] == 'blue':
                color_lower=np.array([100, 43, 46])
                color_upper = np.array([124, 255, 255])
​
        origin_widget.value = bgr8_to_jpeg(frame)
        #cv2.imshow('Capture', frame)
​
        # change to hsv model
        hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
​
        # get mask Use inRange() function and HSV model's blue range upper and lower bounds to get mask. In the mask, the blue parts of the original video will be turned white, others black.
        mask = cv2.inRange(hsv, color_lower, color_upper)
        #cv2.imshow('Mask', mask)
        mask_widget.value = bgr8_to_jpeg(mask)
​
​
        # detect blue Perform bitwise AND operation on mask and original video frame, then the white parts in the mask will be replaced with real images:
        res = cv2.bitwise_and(frame, frame, mask=mask)
        #cv2.imshow('Result', res)
        result_widget.value = bgr8_to_jpeg(res)
​
        time.sleep(0.01)
​
    cap.release()
    #cv2.destroyAllWindows()

After running the code blocks, you can see the camera component display. Due to lighting reasons, recognition effects may vary for different color blocks. You can adjust HSV values, or try adjusting the distance of color blocks, lighting, etc.