Automatic number plate recognition with e-con Systems’ global shutter camera and Raspberry Pi 4

Modern applications like smart surveillance or smart parking lot management are highly dependent on features like vehicle detection, classification, and automatic number plate recognition (ANPR). The algorithms involved in these applications are CPU intensive, which is why they are hosted on cloud servers. However, it also means that such smart solutions require edge AI capabilities to make instant decisions on the edge itself.

In this blog, we’ll look at how you can use e-con’s See3CAM_24CUG, a Full HD global shutter color camera along with Raspberry Pi 4 to provide a Proof-of-Concept (PoC) solution for edge AI-based ANPR applications.

Setting up the hardware

Firstly, let’s quickly go through the hardware (HW) setup.

The hardware setup for an edge Ai based vision system consists of three blocks:

1. Camera
2. Front-end platform
3. Vision Processing Unit (VPU) for edge AI processing

Now, let’s have a detailed look at the modules for each of these blocks.

Camera

Front-end platform

Raspberry Pi 4 is the perfect front-end platform due to its compact form factor, USB 3 capability, and energy efficiency. Also, this PoC runs on the Raspbian OS.

Vision processing unit

Intel® Movidius™ neural computing stick can be leveraged since the Raspberry Pi 4 platform does not have a dedicated VPU. It is simple to use with a USB3 interface and SDK support is available for Raspbian OS.

Below is an image representing the complete hardware setup.

Importance of pre-trained models and demo applications

The next step is to create trained models for your proof of concept. It is important to note that training the models from scratch is time-consuming. So, you can use the pre-trained models provided by the Intel Distribution OpenVINO™ Toolkit. It also contains valuable demo applications for developing a deep learning software. The OpenVINO™ Toolkit kit is referred to as the “Open Model Zoo” – with several object detection models for face detection, person detection, vehicle detection, etc.

In this case, you can use the “vehicle-license-plate-detection-barrier” model and the “security_barrier_camera_demo” as demo applications for vehicle detection, classification, and license plate recognition. The “security_barrier_camera_demo” demo application uses the Caffe framework, OpenCV, and a relevant pre-trained model to run this algorithm.

How to run the ANPR algorithm

Now that you understand which hardware components to use, and how to set them up, let us look at how to run the algorithm for your PoC.

• Flash the Raspberry PI 4 with Raspbian OS.
• Install OpenVINO™ Toolkit for Raspbian* OS (refer https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_raspbian.html)
• Next, you need to build the Open Model Zoo demos on Raspberry Pi 4
  • Clone the Open Model Zoo repository to your home directory
    • Cd ~
    • git clone https://github.com/opencv/open_model_zoo.git
  • Navigate to the demos directory and create a build directory
    • cd ~/open_model_zoo/demos
    • mkdir build && cd build
  • Configure cmake to build the demos for Raspberry Pi 4
    • cmake -DCMAKE_BUILD_TYPE=Release ..
  • Run ‘make help’ to list available options
    • make help
  • Run ANPR with movidius in raspberry PI
    • ./security_barrier_camera_demo -i <camera_path> -m <path_to_model>/vehicle-license-plate-detection-barrier-0106.xml -m_va <path_to_model>/vehicle-attributes-recognition-barrier-0039.xml -m_lpr <path_to_model>/license-plate-recognition-barrier-0001.xml -d MYRIAD -d_va MYRIAD -d_lpr MYRIAD

See3CAM_24CUG: Ensuring superior performance for ANPR

As mentioned earlier, this algorithm provides details, such as the vehicle type (car, truck, etc.), vehicle color, and license plate recognition as outputs. But its accuracy and frame rate performance depend on the pre-trained model and the camera output quality. e-con’s See3CAM_24CUG can capture high-quality images of moving objects without any motion blur or rolling shutter artifacts. Moreover, it has been proven to achieve 90% -93% accuracy and up to 15 fps of frame rate for this PoC.

Below is a sample image showing the results of running the ANPR algorithm using See3CAM_24CUG as the camera.

Here, we have chosen a USB camera (See3CAM_24CUG) to keep things simple for this PoC. e-con Systems also has a MIPI-CSI2 version of the same color global shutter camera (e-CAM20_CURB) for the Raspberry Pi 4 kit.

This POC can work with the MIPI camera module with just a few minor changes.

Related cameras

Below are the global shutter cameras we have discussed in this article.

1. See3CAM_24CUG – Full HD Global Shutter USB 3.1 Gen1 Color Camera
2. e-CAM20_CURB – Full HD Global Shutter MIPI Camera solution for Raspberry Pi 4
3. eCAM24_CUNX – Global shutter Camera for NVIDIA® Jetson Xavier™ NX / TX2 NX / Nano
4. e-CAM25_CUXVR – Six Synchronized Global Shutter Cameras for Jetson AGX Xavier™
5. STURDeCAM25_CUXVR – IP67 GMSL2 Global Shutter camera for Jetson AGX Xavier™

Related Videos

1. https://www.youtube.com/watch?v=nCaN9LarqSA