image processing - issue of the recognize people by their clothes color with not severe illumination environments

Question

I am interested in the human following using a real robot. I'd like to use the color of clothes as a key feature to identify the target person in front of the robot to follow him/ her but I am suffering due to it is a weak feature with a very simple illumination changing. So, I need to alter this algorithm to another or update values (RGB) online in real-time but I don't have enough experience with image processing.

this is my full code for color detection:

import cv2
import numpy as np
from imutils.video import FPS
# capturing video through webcam
import time

cap = cv2.VideoCapture(0)

width = cap.get(3)  # float
height = cap.get(4)  # float
print width, height
time.sleep(2.0)
fps = FPS().start()
while (1):
    _, img = cap.read()

    if _ is True:
        hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
    else:
        continue
    # blue color

    blue_lower = np.array([99, 115, 150], np.uint8)
    blue_upper = np.array([110, 255, 255], np.uint8)
    blue = cv2.inRange(hsv, blue_lower, blue_upper)
    kernal = np.ones((5, 5), "uint8")
    blue = cv2.dilate(blue, kernal)
    res_blue = cv2.bitwise_and(img, img, mask=blue)

            # Tracking blue
    (_, contours, hierarchy) = cv2.findContours(blue, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
    for pic, contour in enumerate(contours):
        area = cv2.contourArea(contour)
        if (area > 300):
            x, y, w, h = cv2.boundingRect(contour)
            img = cv2.rectangle(img, (x, y), (x + w, y + h), (255, 0, 0), 2)
            cv2.putText(img, "Blue Colour", (x, y), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (255, 0, 0))
    cv2.imshow("Color Tracking", img)
    if cv2.waitKey(10) & 0xFF == ord('q'):
        cap.release()
        cv2.destroyAllWindows()
        break
    fps.update()

    # stop the timer and display FPS information
    fps.stop()
#    print("[INFO] elapsed time: {:.2f}".format(fps.elapsed()))
#    print("[INFO] approx. FPS: {:.2f}".format(fps.fps()))

these are the outputs:

1- recognize a person by his clothes color

2- it is lost, the illumination changing is a very simple not severe

Any ideas or suggestions will be appreciated

Answer 1

It does look like you need to use a bit more advanced color similarity function to handle complex cases. Delta E will be the right starting point.

Proper threshold or several colors with associated thresholds will help to achieve pretty accurate results:

See the list of colours on the right side

Complete example.