How embedded vision is contributing to the AI revolution in autonomous tractors

Ever since John Deere helped mechanize agriculture for the first time back in 1837 with the first commercially successful steel plow, tractors have evolved and come a long way. From manually operated vehicles to partially automated tractors to finally fully autonomous systems, tractors are testimony to what automation can do to enable agricultural growth and prosperity. The fast growth of AI capabilities and embedded processing platforms has also fuelled this tectonic shift in the auto farming space.

However, it is not just the artificial intelligence algorithms or IoT (Internet of Things) that act as the catalyst to this revolution. For these technologies to enable intelligent and automated decision making, they need to collect accurate image data. This is where embedded vision comes into play. Embedded camera systems help autonomous tractors capture all the required image data for a farmer to make informed decisions.

In this edition of Markets Monday, we do a deep dive into how OEM cameras and camera modules have helped autonomous tractor manufacturers put AI to its best use to achieve better results in agriculture.

What is an AI-enabled autonomous tractor? What are its functions?

One of the key focus areas of the new era of farming technology is precision agriculture. Precision agriculture is characterized by the use of technology to monitor, analyze, and take actions based on various crop parameters, that too with high precision. Autonomous tractors leverage computer vision, telematics, deep learning algorithms, and cloud-based mobile applications to enable this. While the 1990s saw tractors using GPS technology for navigation, today’s advanced autonomous tractors use technologies such as LiDAR and depth cameras to measure depth, thereby moving autonomously without any human intervention. Cameras are also used for collecting data on plant growth, weed spread, plant diseases, nutrient deficiencies, insect infestations, etc.

The data collected from these cameras are sent to a cloud-based application to derive insights on crop health and calculate parameters such as the NDVI (Normalized Difference Vegetation Index). Once the data is analyzed using machine learning and deep learning algorithms, the insights are fed into the dashboard of a mobile application (for farmers’ viewing) as well as to the tractor for carrying out autonomous tasks. An example of such an autonomous task is spraying pesticides and insecticides only on those parts of the plants that are affected with high levels of precision. Spreading fertilizers is also one of the camera and AI-enabled functions carried out by autonomous tractors.

It goes without saying that autonomous tractors perform all the tasks that a manually operated tractor does, only that they are completely automated and offer much better precision by eliminating the possibility of human errors.

Role of cameras in an autonomous tractor

We already briefly touched upon the functions of embedded cameras in autonomous tractors in the previous sections. Here, we will dive into it a bit more in detail. Predominantly, cameras are used in tractors for 4 purposes:

1. Depth sensing:
2. Obstacle detection
3. Previewing or telestreaming
4. Capturing crop and plant details.

Depth sensing

For depth sensing, LiDAR is the most recommended method owing to its ability to address larger depth ranges (in excess of even 10 meters). However, LiDAR is costly. This is where camera technology can help. Camera technologies such as stereo, time of flight, and structured light help to measure depth for the purposes of navigation, distance calculation, and precisely locating an object or coordinate. They work the best when the depth range is within the limits of 10 meters. While a stereo camera uses two sensors as a stereo pair, time of flight and structured light cameras use the light illumination technique to measure depth.

Obstacle detection

Autonomous tractors need to quickly detect obstacles for ensuring the safety of the pedestrians and objects nearby. The majority of the accidents happen in autonomous tractors when people get closer to the PTO (Power Take-Off) unit. To avoid this, cameras are placed at the read end of the tractor to detect the presence of an object or human near the PTO unit. Cameras also help in identifying which implement is connected to the PTO device.

Previewing and telestreaming

As the term suggests, cameras help to capture and stream video and image data for farmers to consistently monitor crop quality and other parameters. Depending on the use case, a camera module with the necessary features can be chosen for this.

Capturing crop and plant details

This is pretty straightforward. Measuring crop growth and detecting issues require the camera system to be able to capture images and videos with a high level of detail. In some cases, a high resolution camera might be required. If the tractor is moving fast enough to cause rolling shutter artifacts, a global shutter camera is preferred. We will discuss more on the camera features in the next section.

Key features required for a camera system used in autonomous tractors

Building the perfect camera system for an autonomous tractor comes with its own set of challenges. And there is no single camera that can meet all the image capture requirements of an autonomous tractor. A combination of depth and surround view cameras has to be used in most cases. To give you some perspective, a modern autonomous tractor can have as many as 48 cameras in it.

Here, let us look at all the camera features you need to consider while choosing a camera solution for your autonomous tractor.

Depth sensing and accuracy

Autonomous tractors typically do not demand high levels of depth accuracy. However, as discussed before, the depth range to be covered many a time is more than 10 meters, in which case LiDAR is recommended. Or else, a stereo camera, time of flight camera, or structured light camera can be used.

Resolution and digital zoom

In cases where you need to capture minute details, a high resolution camera module with features such as digital zoom might be required.

HDR (High Dynamic Range)

HDR is required to address challenging outdoor lighting conditions. HDR cameras are capable of capturing both the brightest and darkest regions of the same scene without losing the details.

Long distance transmission

Due to their large size, most of the autonomous tractor use cases would need cameras to be placed at a large distance from the host processor. In such a scenario, the camera interface has to support long distance transmission. GMSL cameras are a perfect fit in such situations to transmit image and video data up to a distance of 15 meters from the host processor with very low latency.

Multi-camera synchronization and external trigger support

As mentioned before, an autonomous tractor requires multiple cameras to work in tandem. Having external trigger support would be an added advantage to control camera operation with precision. For instance, e-CAM24_CUNX – Full HD global shutter camera – from e-con Systems comes with both multi-camera synchronization and external trigger support.

Compatibility with ARM platforms

Since an autonomous tractor has to do a lot of image processing for AI based analysis, the cameras used in it should be compatible with the most popular edge AI based processors such as the NVIDIA Jetson, NXP i.MX series, Google Coral, Raspberry Pi, etc.

Shutter type and frame rate

In cases where the tractor has to move fast and might cause rolling shutter artifacts, a global shutter camera module is recommended. To avoid motion blur, a high frame rate camera with low exposure time is recommended.

IP rated enclosure

Tractors have to sometimes operate in harsh weather conditions. To ensure that the camera operates reliably in such conditions, it is a good idea to use a camera with an IP rated enclosure such as the STURDeCAM20 – IP67 Full HD HDR GMSL2 camera – from e-con Systems. This helps to protect the camera system from dust and water.

Antifogging capability

If the tractor has to operate in humid environments, a camera with antifogging glass is a good to have. STURDeCAM25 – IP67 rated Full HD GMSL2 global shutter camera module – from e-con Systems comes with the antifogging feature.

It is to be noted that not all autonomous tractors would require cameras with all the features mentioned above. The choice of camera will depend on your particular use case and the environment where the camera is used. It would be useful to take the help of an imaging partner like e-con Systems who has experience in integrating cameras into autonomous tractors to take you through the journey of camera selection and integration.

Cameras from e-con Systems for autonomous tractors

e-con Systems has worked with multiple autonomous tractor manufacturers to integrate cameras for the purposes of obstacle detection, object identification/classification, depth estimation, etc. Given below are the cameras from e-con Systems which are suitable for autonomous tractors:

• STURDeCAM34 – 3MP AR0341AT 140dB HDR Camera Module

• STURDeCAM31 – 3MP Sony® ISX031 120dB HDR Camera Module

• STURDeCAM20 – IP67 rated Full HD HDR GMSL2 camera
• STURDeCAM21 – IP67 AR0233 Full HD GMSL2 HDR Camera with LFM
• NileCAM20 – Full HD HDR GMSL2 camera
• STURDeCAM25 – IP67 rated Full HD GMSL2 global shutter camera module
• NileCAM25 – Full HD GMSL2 global shutter camera module
• NileCAM81 – 4K AR0821 HDR GMSL2 camera

Hope this blog threw some light into how cameras and embedded vision play a key role in the AI-enabled autonomous tractor revolution. If you need any help in picking and integrating the right cameras into your autonomous tractors, please write to us at camerasolutions@e-consystems.com. You could also visit our Camera Selector to get a full view of our camera portfolio.