Automatic number plate recognition with e-con Systems' global shutter camera and Raspberry Pi 4

Modern applications like smart surveillance or smart parking lot management are highly dependent on features like vehicle detection, classification, and automatic number plate recognition (ANPR). The algorithms involved in these applications are CPU intensive, which is why they are hosted on cloud servers. However, it also means that such smart solutions require edge AI capabilities to make instant decisions on the edge itself.

In this blog, we’ll look at how you can use e-con’s See3CAM_24CUG, a Full HD global shutter color camera along with Raspberry Pi 4 to provide a Proof-of-Concept (PoC) solution for edge AI-based ANPR applications.

Setting up the hardware

Firstly, let’s quickly go through the hardware (HW) setup.

The hardware setup for an edge Ai based vision system consists of three blocks:

Camera
Front-end platform
Vision Processing Unit (VPU) for edge AI processing

Now, let’s have a detailed look at the modules for each of these blocks.

Camera

See3CAM_24CUG is a full HD color global shutter USB 3.1 Gen 1 camera that ensures accurate and fast capture of moving scenes. It features the AR0234CS CMOS image sensor with a pixel size of 3.0 x 3.0 µm and a dedicated high-performance Image Signal Processor (ISP) for performing auto white balance and auto-exposure control. This global shutter camera can capture images at a high frame rate of 120 fps, which helps to minimize frame-to-frame distortion and reduce motion blur. Hence, it produces smoother images with greater scene details while capturing fast-moving objects.

Front-end platform

Raspberry Pi 4 is the perfect front-end platform due to its compact form factor, USB 3 capability, and energy efficiency. Also, this PoC runs on the Raspbian OS.

Vision processing unit

Intel® Movidius™ neural computing stick can be leveraged since the Raspberry Pi 4 platform does not have a dedicated VPU. It is simple to use with a USB3 interface and SDK support is available for Raspbian OS.

Below is an image representing the complete hardware setup.

**Hardware setup: Raspbery Pi 4, Intel® Movidius™ Neural Computing Stick & See3CAM_24CUG USB**

Importance of pre-trained models and demo applications

The next step is to create trained models for your proof of concept. It is important to note that training the models from scratch is time-consuming. So, you can use the pre-trained models provided by the Intel Distribution OpenVINO™ Toolkit. It also contains valuable demo applications for developing a deep learning software. The OpenVINO™ Toolkit kit is referred to as the “Open Model Zoo” – with several object detection models for face detection, person detection, vehicle detection, etc.

In this case, you can use the “vehicle-license-plate-detection-barrier” model and the “security_barrier_camera_demo” as demo applications for vehicle detection, classification, and license plate recognition. The “security_barrier_camera_demo” demo application uses the Caffe framework, OpenCV, and a relevant pre-trained model to run this algorithm.

How to run the ANPR algorithm

Now that you understand which hardware components to use, and how to set them up, let us look at how to run the algorithm for your PoC.

Flash the Raspberry PI 4 with Raspbian OS.
Install OpenVINO™ Toolkit for Raspbian* OS (refer https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_raspbian.html)
Next, you need to build the Open Model Zoo demos on Raspberry Pi 4
- Clone the Open Model Zoo repository to your home directory
  - Cd ~
  - git clone https://github.com/opencv/open_model_zoo.git
- Navigate to the demos directory and create a build directory
  - cd ~/open_model_zoo/demos
  - mkdir build && cd build
- Configure cmake to build the demos for Raspberry Pi 4
  - cmake -DCMAKE_BUILD_TYPE=Release ..
- Run ‘make help’ to list available options
  - make help
- Run ANPR with movidius in raspberry PI
  - ./security_barrier_camera_demo -i <camera_path> -m <path_to_model>/vehicle-license-plate-detection-barrier-0106.xml -m_va <path_to_model>/vehicle-attributes-recognition-barrier-0039.xml -m_lpr <path_to_model>/license-plate-recognition-barrier-0001.xml -d MYRIAD -d_va MYRIAD -d_lpr MYRIAD

See3CAM_24CUG: Ensuring superior performance for ANPR

As mentioned earlier, this algorithm provides details, such as the vehicle type (car, truck, etc.), vehicle color, and license plate recognition as outputs. But its accuracy and frame rate performance depend on the pre-trained model and the camera output quality. e-con’s See3CAM_24CUG can capture high-quality images of moving objects without any motion blur or rolling shutter artifacts. Moreover, it has been proven to achieve 90% -93% accuracy and up to 15 fps of frame rate for this PoC.

Below is a sample image showing the results of running the ANPR algorithm using See3CAM_24CUG as the camera.

**Sample image showing the results of ANPR algorithm**

Here, we have chosen a USB camera (See3CAM_24CUG) to keep things simple for this PoC. e-con Systems also has a MIPI-CSI2 version of the same color global shutter camera (e-CAM20_CURB) for the Raspberry Pi 4 kit.

This POC can work with the MIPI camera module with just a few minor changes.

Related cameras

Below are the global shutter cameras we have discussed in this article.

Automatic number plate recognition with e-con Systems’ global shutter camera and Raspberry Pi 4

Smart surveillance devices in parking lots and smart traffic systems need to read license plates to identify vehicles. See how you can use See3CAM_24CUG, e-con Systems’ Full HD global shutter color camera, and Raspberry Pi 4 to build a PoC solution for edge AI-based ANPR devices.