Parking Slot Detection using MaskRCNN

Detecting the parking slot occupancy can help us by providing the exact location of the place which is empty for parking and save our time. For this purpose, we can use Image Detection to get the location of the parking space which is empty for parking.
Since there are many online models available for the detection of vehicles, like MaskRCNN, YOLO, etc, on the basis of our requirement we can use any of these.
Requirements can be based on precision, accuracy or F1-scores which means every model has its own benefit. YOLO is very faster and can detect the object in a very small period of time but on the other hand it has a lower accuracy score. MaskRCNN is slower than YOLO but its accuracy score is greater than YOLO.
For our purpose the slower model will work if it is giving us better accuracy. Thus we have used MaskRCNN for detecting the vehicle/empty parking slots.

System Requirement

For processing the video frames in real-time we need a good system that can handle at least 60 frames per minute.

~CPU: Intel i7 7700K (The letter K after the number denotes that the processor is designed for performance. It also means the chip is “unlocked,” where its clock speed can be overclocked by a user thus it is a must. And higher the number after 7xxx means better the processor)

~GPU: if you are planning to run your model on GPU then, use Nvidia GTX 1050Ti at least or you can go higher to GTX 2080Ti. NOTE:- For this task the GPUs are a must, the computation power of CPU is not enough for us.

~RAM: 8 GB with SSD (take 16GB as swap) or 16GB with HDD (16GB swap)

Enabling GPU processing:

Since we are processing the video, it requires high computation power thus the CPU is not enough for us and we use GPUs to do our work.For enabling the GPU just follow the below steps.

->Detect and install the Nvidia-GPU driver.

->Install CUDA (this is the Nvidia’s library/API package for programming in GPU)

->Install cuDNN (it is the library for deep neural networks built using CUDA).

Installing Nvidia_GPU in Ubuntu

-Type the following command in terminal it will give you the information about your GPU.

-This must show an output similar to this,

-Go to this link to download the driver for the corresponding GPU. search the driver and download it.

-For Ubuntu, it will give you a .run file, type the following command to make the script runnable.

-Disable the default Nouveau NVIDIA driver, type the following commands in terminal:

-To check if both commands have run in the right way just type the following command:

-The output should be like

-Now run the following command

-Now install the Nvidia driver which we have downloaded with the command

-GPU must be working now

Setting the Python CUDA toolkit

~Go to the Nvidia website and find the 10.2 version of the Cuda toolkit for the respective os.

~Now download and run the .run file (for ubuntu).

~follow command prompt

~Now reboot the system

~Run the following command to see if Cuda is installed or not:

~The output will be similar to:

Installing CuDNN:

->Go to this link and download the suitable version of cuDNN (we are using cuDNN 7.4).

->Follow the instructions written here.

->Now add /usr/local/cuda/lib64 to path or type,

Checking if GPU is enabled

->Activate your virtualenv (if you’re using a virtual environment for your project)

->Open python shell and Type the command :

->must give output like

How parking detection is working

It will use Mask R- CNN( it is a deep neural model that efficiently detects objects in an image while simultaneously generating a high-quality segmentation mask for each instance ) for detecting the vehicles in the parking slots. If there is an empty space in the parking area then it will detect it and show a green square otherwise it will show a red box for filled space. On the upper side of the screen, it will show the number of parking slots that are empty.

Approach to make it work:

->We will firstly take a video or image from the CCTV-Camera installed in the parking space.

->Building parking slots (custom bounding boxes where cars must be parked.

1. Make boxes based on the parking area with coordinates:
2. Name,truncated, difficult, bndbox/xmin, bndbox/ymin, bndbox/xmax,bndbox/ymax
3. Bndbox = bounding box which is our ROI
4. Xmax, xmin, ymin, ymax are the maximum and minimum values of x and y coordinates of the respective bounding boxes.
5. The name denotes the class of vehicle car/truck/bus etc.
6. Make a CSV file with the respective columns and save it.

->From our program we will try to detect the vehicles in the parking space and check if the parking space is occupied by seeing if any vehicle overlaps IoU(Intersection over union).

->Draw the red box at all occupied spaces and draw the green boxes around unoccupied spaces.

Tool for building parking slot coordinates:

~Go to https://dataturks.com/ and upload the frames extracted from running the script Extract_frames.ipynb.

~Now make bounding boxes and download the JSON.

~Run the script json_to_csv_converter.ipynb.

~This will create a CSV file with parking slot coordinates.

Using our pre-built model for parking slot detection:

Download the GIT repo from here.

~Setup environment using the commands:

->Clone this repository: https://github.com/matterport/Mask_RCNN.git

->Change the working directory to MaskRCNN

->Install dependencies using

->Run setup from the repository root directory

->Download pre-trained COCO weights (mask_rcnn_coco.h5) from here. And place it inside our git repo root directory.

->Install pandas, scipy and open-cv tf and keras using:

->Now run the command

Code overview

~The file Extract_frame.ipynb contains the code to extract 10 frames from a video for our help and saves those frames to the test folder. We can use those frames to find the bounding box location

~The file Newtry2.py contains our model which will do our work for parking slot detection. Let’s go through this file.

Since there are many classes that are detected by MaskRCNN but we need to identify only vehicles. As we can see in the below classes, for us only class index 3,6,8 are useful.

We have used get_car_boxes function to do our work. 

class_names = [‘BG’, ‘person’, ‘bicycle’, ‘car’, ‘motorcycle’, ‘airplane’, ‘bus’, ‘train’, ‘truck’, ‘boat’, ‘traffic light’,’fire hydrant’, ‘stop sign’, ‘parking meter’, ‘bench’, ‘bird’,’cat’, ‘dog’, ‘horse’, ‘sheep’, ‘cow’, ‘elephant’, ‘bear’,’zebra’, ‘giraffe’, ‘backpack’, ‘umbrella’, ‘handbag’, ‘tie’,’suitcase’, ‘frisbee’, ‘skis’, ‘snowboard’, ‘sports ball’,’kite’, ‘baseball bat’, ‘baseball glove’, ‘skateboard’,’surfboard’, ‘tennis racket’, ‘bottle’, ‘wine glass’, ‘cup’,’fork’, ‘knife’, ‘spoon’, ‘bowl’, ‘banana’, ‘apple’,’sandwich’, ‘orange’, ‘broccoli’, ‘carrot’, ‘hot dog’, ‘pizza’,’donut’, ‘cake’, ‘chair’, ‘couch’, ‘potted plant’, ‘bed’,’dining table’, ‘toilet’, ‘tv’, ‘laptop’, ‘mouse’, ‘remote’,’keyboard’, ‘cell phone’, ‘microwave’, ‘oven’, ‘toaster’,’sink’, ‘refrigerator’, ‘book’, ‘clock’, ‘vase’, ‘scissors’,’teddy bear’, ‘hair drier’, ‘toothbrush’]

Demo of MaskRCNN detection of cars