Machine Learning Application Part 1 - Number Plate Recognition system

1. Introduction

In this article I am going to share the steps involved in Number Plate Recognition System considering the input as image and this system can be extended to process the real time video feed.

In the upcoming articles I am going to write about the various real time applications of Machine learning algorithms and their Implementation.

2. Number Plate Recognition system :

It is one of the the real time application of machine learning and Image processing techniques.This system is the state of the art Optical Character Recognition (OCR) with the help of OpenCV and EasyOCR libraries. This system has multiple real world applications like Traffic control, Parking lots access control, Traffic Violation Detection ,Traffic violation penalisation and Electronic toll collection.

3. Steps Involved in Optical Character Recognition (OCR):

There are six primary algorithms that the software requires for identifying a license plate:

Plate localization – responsible for finding and isolating the plate on the picture
Plate orientation and sizing – compensates for the skew of the plate and adjusts the dimensions to the required size
Normalization – adjusts the brightness and contrast of the image
Character segmentation – finds the individual characters on the plates
Optical character recognition
Syntactical/Geometrical analysis – check characters and positions against country-specific

4. Implementation:

4.1 Step 1-Installing and Importing Libraries:

The required Libraries/dependencies are installed and imported in this step.

!pip install easyocr
!pip install imutils

import cv2
from matplotlib import pyplot as plt
import numpy as np
import imutils
import easyocr

4.2 Step 2 : Read in Image ,GrayScale and Blur

After reading the image, Gray scaling is performed which is common in all image processing steps. This speeds up other following process since we no longer have to deal with the color details when processing an image.

img = cv2.imread("/content/CAR_LP.jpeg")
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
plt.imshow(cv2.cvtColor(gray,cv2.COLOR_BGR2RGB))

The image would be transformed something like this when this step is done.

4.3 Step 3: Apply filter and find edges for localization:

Using a bilateral filter (Bluring) will remove the unwanted details from an image.You can increase the sigma color and sigma space from 17 to higher values to blur out more background information, but be careful that the useful part does not get blurred.

bfilter = cv2.bilateralfilter(gray, 11, 17, 17) # Noise reduction

The next step is interesting where we perform edge detection. There are many ways to do it, the most easy and popular way is to use the canny edge method from OpenCV. The Threshold Value 30 and Threshold Value 200 are the minimum and maximum threshold values. Only the edges that have an intensity gradient more than the minimum threshold value and less than the maximum threshold value will be displayed.

edged = cv2.Canny(bfilter, 30, 200) #Edge detection
plt.imshow(cv2.cvtColor(edged,cv2.COLOR_BGR2RGB))

4.4 Step 4: Find Contours and Apply Mask:

Now we can start looking for contours on our image.

Once the counters have been detected we sort them from big to small and consider only the first 10 results ignoring the others. In our image the counter could be anything that has a closed surface but of all the obtained results the license plate number will also be there since it is also a closed surface.

keypoints = cv2.findContours(edged.copy(), cv.RETR_TREE, cv2.CCHAIN_APPROX_SIMPLE)
contours = imutils.grab_contours(keypoints)
contours = sorted(contours, key=cv2.contourArea, reverse=True)[:10]

To filter the license plate image among the obtained results, we will loop though all the results and check which has a rectangle shape contour with four sides and closed figure. Since a license plate would definitely be a rectangle four sided figure.

location = None
for contour in contours:
      approx = cv2.approxPolyDP(contour, 10, True)
      if len(approx) ==4:
            location = approx
            break

location

Now that we know where the number plate is, the remaining information is pretty much useless for us. So we can proceed with masking the entire picture except for the place where the number plate is.

mask = np.zeros(gray.shape, np.unit8)
new_image = cv2.drawContours(mask, [location], 0, 255, -1)
new_image = cv2.bitwise_and(img, img, mask=mask)

plt.imshow(cv2.cvtColor(new_image ,cv2.COLOR_BGR2RGB))

Then Character segmentation is performed where use this image to detect the character in it. The below code crops the roi (Region of interest) image form the main image

(x,y) = np.where(mask++255)
(x1,y1) = (np.min(x) ,np.min(y))
(x2,y2) = (np.max(x) ,np.max(y))
cropped_image = gray[x1:x2+1, y1:y2+1]

plt.imshow(cv2.cvtColor(cropped_image ,cv2.COLOR_BGR2RGB))

4.5 Step 5: Use Easy OCR To Read Text:

Next step is to actually read the number plate information from the segmented image. Here I have used EasyOCR to read characters from image.

reader = easyocr.Reader(['en'])
result = reader.readtext(cropped_image)
result

4.6 Step 6- Render Result:

Then as a final step we are visualising the detected Licence Number around the actual image.

text = result[0][-2]
font = cv2.Font_HERSHEY_SIMPLEX
res = cv2.putText(img, text=text, org=(approx[0][0][0], approx[1][0][1]+60), fontFace=font, fonyScale=1, colour=(0,255,0),
thickness=2, lineType=cv2.Line_AA)
res = cv2.rectangle(img, tuple(approx[0][0]), tuple(approx[2][0]), (0,255,0), 3)
plt.imshow(cv2.cvtColor(res,cv2.COLOR_BGR2RGB))

5 Difficulties in the Implementation:

There are a number of possible difficulties that the software must be able to cope with. These include:

Poor file resolution, usually because the plate is too far away but sometimes resulting from the use of a low-quality camera
Blurry images, particularly motion blur
Poor lighting and low contrast due to overexposure, reflection or shadows
An object obscuring (part of) the plate, quite often a tow bar, or dirt on the plate
Read license plates that are different at the front and the back because of towed trailers, campers, etc.
Vehicle lane change in the camera's angle of view during license plate reading
A different font, popular for vanity plates (some countries do not allow such plates, eliminating the problem)
Circumvention techniques
Lack of coordination between countries or states. Two cars from different countries or states can have the same number but different design of the plate.

While some of these problems can be corrected within the software, it is primarily left to the hardware side of the system to work out solutions to these difficulties. Increasing the height of the camera may avoid problems with objects (such as other vehicles) obscuring the plate but introduces and increases other problems, such as adjusting for the increased skew of the plate.