Home/Parking Slot Detection using MaskRCNN

Parking Slot Detection using MaskRCNN

Published On: 10 April 2020.By .
  • General

Detecting the parking slot occupancy can help us by providing the exact location of the place which is empty for parking and save our time. For this purpose, we can use Image Detection to get the location of the parking space which is empty for parking.
Since there are many online models available for the detection of vehicles, like MaskRCNN, YOLO, etc, on the basis of our requirement we can use any of these.
Requirements can be based on precision, accuracy or F1-scores which means every model has its own benefit. YOLO is very faster and can detect the object in a very small period of time but on the other hand it has a lower accuracy score. MaskRCNN is slower than YOLO but its accuracy score is greater than YOLO.
For our purpose the slower model will work if it is giving us better accuracy. Thus we have used MaskRCNN for detecting the vehicle/empty parking slots.

System Requirement

For processing the video frames in real-time we need a good system that can handle at least 60 frames per minute.

~CPU: Intel i7 7700K (The letter K after the number denotes that the processor is designed for performance. It also means the chip is “unlocked,” where its clock speed can be overclocked by a user thus it is a must. And higher the number after 7xxx means better the processor)

~GPU: if you are planning to run your model on GPU then, use Nvidia GTX 1050Ti at least or you can go higher to GTX 2080Ti. NOTE:- For this task the GPUs are a must, the computation power of CPU is not enough for us.

~RAM: 8 GB with SSD (take 16GB as swap) or 16GB with HDD (16GB swap)

Enabling GPU processing:

Since we are processing the video, it requires high computation power thus the CPU is not enough for us and we use GPUs to do our work.For enabling the GPU just follow the below steps.

->Detect and install the Nvidia-GPU driver.

->Install CUDA (this is the Nvidia’s library/API package for programming in GPU)

->Install cuDNN (it is the library for deep neural networks built using CUDA).

Installing Nvidia_GPU in Ubuntu

-Type the following command in terminal it will give you the information about your GPU.

ubuntu-drivers devices

-This must show an output similar to this,

Terminal Snapshot

-Go to this link to download the driver for the corresponding GPU. search the driver and download it.

Select the compatible driver

-For Ubuntu, it will give you a .run file, type the following command to make the script runnable.

sudo apt-get install gcc
sudo apt-get install make

-Disable the default Nouveau NVIDIA driver, type the following commands in terminal:

sudo bash -c "echo blacklist nouveau > /etc/modprobe.d/blacklist-nvidia-nouveau.conf"
sudo bash -c "echo options nouveau modeset=0 >> /etc/modprobe.d/blacklist-nvidia-nouveau.conf"

-To check if both commands have run in the right way just type the following command:

cat /etc/modprobe.d/blacklist-nvidia-nouveau.conf

-The output should be like

Blacklist nouveau
Options nouveau set to 0

-Now run the following command

sudo update-initramfs -u
sudo reboot

-Now install the Nvidia driver which we have downloaded with the command

sudo bash NVIDIA-Linux-x86_64-410.73.run

sudo reboot

-GPU must be working now

Setting the Python CUDA toolkit

~Go to the Nvidia website and find the 10.2 version of the Cuda toolkit for the respective os.

~Now download and run the .run file (for ubuntu).

sudo bash <file_name>.run 

~follow command prompt

~Now reboot the system

~Run the following command to see if Cuda is installed or not:

nvidia -smi

~The output will be similar to:

Installing CuDNN:

->Go to this link and download the suitable version of cuDNN (we are using cuDNN 7.4).

->Follow the instructions written here.

->Now add /usr/local/cuda/lib64 to path or type,

export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

Checking if GPU is enabled

->Activate your virtualenv (if you’re using a virtual environment for your project)

->Open python shell and Type the command :

import tensorflow as tf

tf.test.is_gpu_available(cuda_only=False,
                             min_cuda_compute_capability=None)

->must give output like

How parking detection is working

It will use Mask R- CNN( it is a deep neural model that efficiently detects objects in an image while simultaneously generating a high-quality segmentation mask for each instance ) for detecting the vehicles in the parking slots. If there is an empty space in the parking area then it will detect it and show a green square otherwise it will show a red box for filled space. On the upper side of the screen, it will show the number of parking slots that are empty.

Approach to make it work:

->We will firstly take a video or image from the CCTV-Camera installed in the parking space.

->Building parking slots (custom bounding boxes where cars must be parked.

  1. Make boxes based on the parking area with coordinates:
  2. Name,truncated, difficult, bndbox/xmin, bndbox/ymin, bndbox/xmax,bndbox/ymax
  3. Bndbox = bounding box which is our ROI
  4. Xmax, xmin, ymin, ymax are the maximum and minimum values of x and y coordinates of the respective bounding boxes.
  5. The name denotes the class of vehicle car/truck/bus etc.
  6. Make a CSV file with the respective columns and save it.

->From our program we will try to detect the vehicles in the parking space and check if the parking space is occupied by seeing if any vehicle overlaps IoU(Intersection over union).

->Draw the red box at all occupied spaces and draw the green boxes around unoccupied spaces.

Tool for building parking slot coordinates:

~Go to https://dataturks.com/ and upload the frames extracted from running the script Extract_frames.ipynb.

~Now make bounding boxes and download the JSON.

~Run the script json_to_csv_converter.ipynb.

~This will create a CSV file with parking slot coordinates.

Using our pre-built model for parking slot detection:

Download the GIT repo from here.

~Setup environment using the commands:

virtualenv venv #this will create a virtual environment
source venv/bin/activate

->Clone this repository: https://github.com/matterport/Mask_RCNN.git

->Change the working directory to MaskRCNN

->Install dependencies using

pip install -r requirements.txt

->Run setup from the repository root directory

Python setup.py install

->Download pre-trained COCO weights (mask_rcnn_coco.h5) from here. And place it inside our git repo root directory.

->Install pandas, scipy and open-cv tf and keras using:

Pip install scipy pandas python-opencv tensorflow==1.15.2 keras

->Now run the command

python New_try2.py to run our model.

Code overview

~The file Extract_frame.ipynb contains the code to extract 10 frames from a video for our help and saves those frames to the test folder. We can use those frames to find the bounding box location

~The file Newtry2.py contains our model which will do our work for parking slot detection. Let’s go through this file.

Since there are many classes that are detected by MaskRCNN but we need to identify only vehicles. As we can see in the below classes, for us only class index 3,6,8 are useful.

We have used get_car_boxes function to do our work. 

class_names = [‘BG’, ‘person’, ‘bicycle’, ‘car’, ‘motorcycle’, ‘airplane’, ‘bus’, ‘train’, ‘truck’, ‘boat’, ‘traffic light’,’fire hydrant’, ‘stop sign’, ‘parking meter’, ‘bench’, ‘bird’,’cat’, ‘dog’, ‘horse’, ‘sheep’, ‘cow’, ‘elephant’, ‘bear’,’zebra’, ‘giraffe’, ‘backpack’, ‘umbrella’, ‘handbag’, ‘tie’,’suitcase’, ‘frisbee’, ‘skis’, ‘snowboard’, ‘sports ball’,’kite’, ‘baseball bat’, ‘baseball glove’, ‘skateboard’,’surfboard’, ‘tennis racket’, ‘bottle’, ‘wine glass’, ‘cup’,’fork’, ‘knife’, ‘spoon’, ‘bowl’, ‘banana’, ‘apple’,’sandwich’, ‘orange’, ‘broccoli’, ‘carrot’, ‘hot dog’, ‘pizza’,’donut’, ‘cake’, ‘chair’, ‘couch’, ‘potted plant’, ‘bed’,’dining table’, ‘toilet’, ‘tv’, ‘laptop’, ‘mouse’, ‘remote’,’keyboard’, ‘cell phone’, ‘microwave’, ‘oven’, ‘toaster’,’sink’, ‘refrigerator’, ‘book’, ‘clock’, ‘vase’, ‘scissors’,’teddy bear’, ‘hair drier’, ‘toothbrush’]

def get_car_boxes(boxes, class_ids):
   car_boxes = []
 
   for i, box in enumerate(boxes):
       # If the detected object isn't a car / truck, skip it
       if class_ids[i] in [3, 8, 6]:
           car_boxes.append(box)
 
   return np.array(car_boxes)
#The class MaskRCNNConfig is used for configuring the MaskRCNN model:
# Configuration that will be used by the Mask-RCNN library

NAME = "coco_pretrained_model_config"
IMAGES_PER_GPU = Number of Images/Frames per GPU per TIME.
GPU_COUNT = Maximum number of GPU we wanna use(it will only use available GPU’s)
NUM_CLASSES = total number of classes which can be classified by the model, there are 80+1(background)
DETECTION_MIN_CONFIDENCE = Minimum confidence for classification
#There are some static variables which are used in our model they are #as follows:
ROOT_DIR = Root directory of the project
MODEL_DIR = Directory to save logs and trained model
COCO_MODEL_PATH = Local path to trained weights file
IMAGE_DIR = Directory of images to run detection on
VIDEO_SOURCE = Video file or camera to process - set this to 0 to use your webcam instead of a video file
# Now Let’s Create a Mask-RCNN model in inference mode
 
config = tensorflow.ConfigProto()
config.inter_op_parallelism_threads = 1 # number of threads we wanna apply
keras.backend.set_session(tensorflow.Session(config=config))
model = MaskRCNN(mode="inference", model_dir=MODEL_DIR, config=MaskRCNNConfig())
#Loading pre-trained model
model.load_weights(COCO_MODEL_PATH, by_name=True)

Demo of MaskRCNN detection of cars

Related content

We Love Conversations

Say Hello
Go to Top