Accessing See3CAM Custom Format with OpenCV

Brief on OpenCV

In the blog Working with See3CAM and OpenCV the basic procedure to access See3CAM USB 3.0 cameras with OpenCV is been discussed. In this blog the procedures and changes to be done to access custom formats supported by See3CAM USB cameras (See3CAM_CU51 supports Y16, See3CAM_10CUG_C supports BY8) with OpenCV is discussed.

This was a question to most of the customers who bought industrial machine vision USB cameras. e-con Systems™’ machine vision camera is designed to get full access to the raw camera data, say for eg, See3CAM_CU51 supports high resolution 12 bit monochrome data which is an important feature in analyzing high gradient monochrome data. OpenCV does not have capability to access custom formats like BY8, Y16 by native.

OpenCV makes use of DirectShow (for Windows) and V4L2 (for Linux) framework internally to access frames from camera. As algorithm developers are concentrated towards optimizing their results, e-con has reduced their effort by providing all necessary patch and modification to OpenCV layer for accessing custom formats provided by e-con USB camera. This will provide user to utilize their current working OpenCV algorithm code with very minimal changes.

Refer to Working with See3CAM and OpenCV blog for installing OpenCV in Windows/Linux. Download the modified OpenCV package for capture interface and sample application source code.

Download link for modified OpenCV binary

Accessing Custom Formats using OpenCV

e-con’s See3CAM_12CUNIR(Y16), See3CAM_CU51(Y16), See3CAM_CU40(Y16), See3CAM_10CUG_C(BY8) and Tara(Y16) supports custom format as of now. The below sample code demonstrates the basic access of the camera’s using OpenCV. The camera is by default set to 1280 x 720 resolution.

To access Tara camera

The sample source code is given in the Tara SDK provided along with the camera.

To use See3CAM_CU40 set the define CU40

Sample code to split the RGB & IR Data & display the images is provided. Note that the missing Green component is filled with the value of the nearest neighbor. For display purpose the 10 Bit images are scaled down to 8 Bit.

To use 12-bit Monochrome (Y16) from See3CAM_CU51 and See3CAM_12CUNIR set the define CU51_12CUNIR

12-bit monochrome data can be used directly for analysis purpose, but for demonstrating purpose, the input 12-bit image is converted to 8-bit and displayed.

To use See3CAM_10CUG_C

Sample code by default streams See3CAM_10CUG_C. Register the Bayer Filter provided by e-con in the package. The result image is by default interpolated and then displayed.

/* OpenCV Streaming Sample Custom format - Y16 and BY8
To test the Camera's Set the corresponding defines to 1 (CU40 - RGB IR Camera, CU51_12CUNIR - CU51/ 12CUNIR camera)
For 10CUG make both defines as 0 */

#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/highgui/highgui.hpp"

#include <iostream>
#include <conio.h>

using namespace cv;
using namespace std;

#define CU40	0
#define CU51_12CUNIR	0 //Both CU51 or 12CUNIR

#define ImageWidth	1280
#define	ImageHeight    720

// Actual Data format BGIRR after conversion BGGR - IR is replaced with the G 
//IR data is collected as a separate image
bool ConvertRGIR2RGGB(Mat BayerRGIR, Mat &BayerRGGB, Mat &IRimage)
{
	//Result image after replacing the IR pixel with the G data
	BayerRGGB = BayerRGIR.clone();
	
	//IR data will be half the size of Bayer Image
	IRimage = Mat(BayerRGIR.size().height / 2, BayerRGIR.size().width / 2, CV_8UC1);

	//copying the IR data and replacing the IR data with G
	for (int Row = 0; Row < BayerRGIR.rows; Row += 2)
	{
		for (int Col = 0; Col < BayerRGIR.cols; Col += 2)
		{
                   //Set the IR Data with Nearby Green 
		   BayerRGGB.at<uchar>(Row + 1, Col) = BayerRGIR.at<uchar>(Row, Col + 1);
                   //Set the IR Data 
		   IRimage.at<uchar>(Row / 2, Col / 2) = BayerRGIR.at<uchar>(Row + 1, Col);
		}
	}

	return true;
}

// Main Function
int main()
{	
	cout << "e-con's Sample OpenCV Application to  Custom Formats" << endl;
	cout << endl <<"Demonstrates the working of e-con's Custom Format(Y16 / BY8)";
        cout << "cameras with the modified libraries of OpenCV" << endl << endl;

	char keyPressed;
	VideoCapture _CameraDevice;	
	Mat ResultImage, InputImage;
	Mat BayerFrame8, IRImage, BGRImage;
	
	//Open the device at the ID 0
	_CameraDevice.open(0);

	if( !_CameraDevice.isOpened()) //Check for the device
	{
		cout << endl << "\tCamera Device not Initialised Successfully" << endl << endl;
		cout << endl << "Press any Key to exit the application" << endl << endl;
	
		_getch();
		return 0;
	}

	//Set up the width and height of the camera
	_CameraDevice.set(CV_CAP_PROP_FRAME_WIDTH,  ImageWidth);
	_CameraDevice.set(CV_CAP_PROP_FRAME_HEIGHT, ImageHeight);

	cout << endl << "Press 'Q / q /Esc' key on the image winodw to exit the application" << endl << endl;

	while(1)
	{
		_CameraDevice >> InputImage; //Read the input image

		if(InputImage.empty()) //Check for the vlid image
		{
			cout << "No frame grabbed!!, check whether the camera is free!!" << endl << endl;
			break;
		}

#if CU51_12CUNIR
				
		//Convert to 8 Bit: 
                //Scale the 12 Bit (4096) Pixels into 8 Bit(255) (255/4096)= 0.06226
		convertScaleAbs(InputImage, ResultImage, 0.06226);

		namedWindow("Y16 to Y8", WINDOW_AUTOSIZE);
		imshow("Y16 to Y8", ResultImage);

#elif CU40
	        //Convert to 8 Bit:
                //Scale the 10 Bit (1024) Pixels into 8 Bit(255) (255/1024)= 0.249023
		convertScaleAbs(InputImage, BayerFrame8, 0.249023);

		//Filling the missing G -channel bayer data
		ConvertRGIR2RGGB(BayerFrame8, BayerFrame8, IRImage);
		
		//Actual Bayer format BG but Opencv uses BGR & Not RGB So taking RG Bayer format
		cvtColor(BayerFrame8, BGRImage, COLOR_BayerRG2BGR);

		namedWindow("Camera BGR Frame", WINDOW_AUTOSIZE);
		imshow("Camera BGR Frame", BGRImage);

		namedWindow("Camera IR Frame", WINDOW_AUTOSIZE);
		imshow("Camera IR Frame", IRImage);

#else //10CUG and other camera's

		namedWindow("Camera Frame", WINDOW_AUTOSIZE);
		imshow("Camera Frame", InputImage);
#endif

		keyPressed = waitKey(1); //Waits for a user input to quit the application

		if(keyPressed == 27 || keyPressed == 'q'  || keyPressed == 'Q' )
		{
			destroyAllWindows();
			break;
		}
	}

	//Release the devices
	_CameraDevice.release();
	
	cout << endl << "Press any Key to exit the application" << endl << endl;	
	_getch();
	
	return 1;
}

With this blog, it is been made clear to access e-con’s custom format Camera’s with OpenCV. The modified OpenCV binary provided for download is built using Visual Studio 2012. For installation instructions of OpenCV with Custom format in Windows / Linux refer to the document provided in the package.

Related posts

e-con Systems Launches See3CAM_CU83 – A 4K RGB-IR USB Camera Delivering Stunning Visuals Day and Night

e-con Systems Offers a Patented Hot-Pluggable GMSL Camera System for Uninterrupted Host Processor Streaming

e-con Systems Introduces STURDeCAM34: The World’s First 140dB HDR Hot Pluggable GMSL2 Camera Targeting Automotive Applications

21 Comment

Lucy March 7, 2016 - 4:45 pm

How can I access the format for See3CAM_CU40 ? Thanks a lot!

Dongsu Kim May 31, 2017 - 12:27 pm

Hi, our company bought your See3CAM_130 cameras and tested your cameras with e-cam view program. The test result was good for our use especially for aufocus function with an ROI-setting function. Problem is we have to use a separate program in which a 13 meagapixel image is captured from your cameras and the captured image should be processed to obtain further information from the image. We use Windows 7 OS and C++ language under visual studio 2010. Can you provide us a sample code for this use?

Anitha Jothiprakash June 1, 2017 - 11:21 am

See3CAM_130 is a UVC camera and can be accessed using directshow or any open source platforms like OpenCV etc.

Madalina Savu July 19, 2017 - 6:08 pm

Hi,
I’m using the See3CAM_CU40 camera. I managed to get the two images using the code from above with OpenCV 2.4.9 and Visual Studio 2013. Can you provide the OpenCV binary for debug mode?
Thank you!

Anitha Jothiprakash July 25, 2017 - 12:39 pm

We don’t have the Open binary for debug mode. If you want Open binary for debug mode you have to create by your self with help of our document given in our blog. For further information, please contact camerasolutions@e-consystems.com

Rui Ponte December 30, 2017 - 7:04 am

Hello I am trying to use the See3CAM_CU40 with cv2 in python inside of a flask web app but I haven’t been able to get it to work. It either crashes, returns an array of all zeros that can’t be further processed by cv2, or returns none-type. What do I need to do to get it working in python either through cv2.VideoCapture() or a direct serial communication?

Anitha Jothiprakash January 3, 2018 - 1:44 pm

Please create the support ticket for your queries. One of our technical expert engineer will discuss further with you over the ticket thread.
https://www.e-consystems.com/create-ticket.asp

Huang Wenjun March 6, 2018 - 1:16 pm

I would like to access data from the camera when a new frame is available, based on the frame rate set, instead of grabbing the data from the camera, as the data that I grabbed could be the previous frame, or several frames from the previous frame. I would like to grab each new frame from the camera. Is it possible?

May I know what is the readout time for 1 line of the CMOS camera?

Anitha Jothiprakash March 15, 2018 - 1:57 pm

Could you please confirm whether you would like to decide when you have to capture the images instead of continuous capturing? If yes, then this is nothing but triggering. There are triggering options available with couple of our cameras. Please confirm if this is your requirement. Also the readout time varies based on cameras. So in which camera you want the readout time .

Tamas May 12, 2018 - 12:49 am

Is it possible to access 1080p video stream with OpenCV?

Anitha Jothiprakash May 23, 2018 - 6:33 pm

Yes, it’s possible to access 1080p video stream with OpenCV.

SeungKye Go August 25, 2018 - 6:58 am

Hi, i buy the CU40, and i want c++ with win10.
I using this code, but he say, ‘No frame grabbed!!, check theter the camera is free!!’.
I don’t know this Problem.

Anitha Jothiprakash August 28, 2018 - 5:54 pm

We will create the support ticket for your issue. One of our technical expert engineers will discuss further with you over the ticket thread.

Josh September 3, 2018 - 1:04 am

Hi,

I’ve made some adjustments to your OpenCV patch to include support for cameras which optionally support Y16 (currently setting fourcc codes with the OpenCV DShow backend is basically broken).

I would like to submit an official pull request, because this will benefit a lot of machine vision users. Can I include your patch code in the pull request? (Otherwise I’m happy to re-implement it, but the solution will be largely similar!).

Best,
Josh

Anitha Jothiprakash September 4, 2018 - 7:34 pm

Please proceed as you requested.

アシュラフ September 27, 2018 - 7:29 am

I am planning to buy this camera, but I would like to check if the provided code work with the latest version of OpenCV (>3) and Visual Studio (2015)

Anitha Jothiprakash September 27, 2018 - 7:37 pm

We have developed sample aplication in OpenCV with C++ and tested our USB camera in visual studio 2012, 2013, 2015. In all these versions we are able to compile the code. If you have followed the document correctly then you will not get any error. If you face any issue in the future, feel free to reach us at techsupport@e-consystems.com. We will assists you in all the best possible ways.

アシュラフ September 28, 2018 - 7:19 am

How about python API?
Does this code when converted to python still valid?

Anitha Jothiprakash October 4, 2018 - 1:08 pm

We haven’t tested our camera in OpenCV python. But there is a similar API available for python.
Please refer the below link and port it your code
https://gist.github.com/bhive01/9a06c0736e6a416af6eebac3f5026728

Siddharth R May 18, 2020 - 5:57 pm

Hi, I am using see3CAM_CU55_MH and trying to capture a 12bit image using Opencv (using Python 3.4). But I am getting 8bit images only.
“set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter.fourcc(‘Y’,’1′,’2′,’ ‘))” I’m using this code for conversion. Is there any other way i can get 12bit images using Opencv and python 3.4

Anitha Jothiprakash May 20, 2020 - 12:01 pm

We are having an option to take Y12 frames using the OpenCV. For Python, it was in development.

Please refer the below link: https://github.com/econsystems/opencv

Add Comment