Brief on OpenCV
In the blog Working with See3CAM and OpenCV the basic procedure to access See3CAM USB 3.0 cameras with OpenCV is been discussed. In this blog the procedures and changes to be done to access custom formats supported by See3CAM USB cameras (See3CAM_CU51 supports Y16, See3CAM_10CUG_C supports BY8) with OpenCV is discussed.
This was a question to most of the customers who bought industrial machine vision USB cameras. e-con Systems™’ machine vision camera is designed to get full access to the raw camera data, say for eg, See3CAM_CU51 supports high resolution 12 bit monochrome data which is an important feature in analyzing high gradient monochrome data. OpenCV does not have capability to access custom formats like BY8, Y16 by native.
OpenCV makes use of DirectShow (for Windows) and V4L2 (for Linux) framework internally to access frames from camera. As algorithm developers are concentrated towards optimizing their results, e-con has reduced their effort by providing all necessary patch and modification to OpenCV layer for accessing custom formats provided by e-con USB camera. This will provide user to utilize their current working OpenCV algorithm code with very minimal changes.
Refer to Working with See3CAM and OpenCV blog for installing OpenCV in Windows/Linux. Download the modified OpenCV package for capture interface and sample application source code.
Download link for modified OpenCV binary
Accessing Custom Formats using OpenCV
e-con’s See3CAM_12CUNIR(Y16), See3CAM_CU51(Y16), See3CAM_CU40(Y16), See3CAM_10CUG_C(BY8) and Tara(Y16) supports custom format as of now. The below sample code demonstrates the basic access of the camera’s using OpenCV. The camera is by default set to 1280 x 720 resolution.
To access Tara camera
The sample source code is given in the Tara SDK provided along with the camera.
To use See3CAM_CU40 set the define CU40
Sample code to split the RGB & IR Data & display the images is provided. Note that the missing Green component is filled with the value of the nearest neighbor. For display purpose the 10 Bit images are scaled down to 8 Bit.
To use 12-bit Monochrome (Y16) from See3CAM_CU51 and See3CAM_12CUNIR set the define CU51_12CUNIR
12-bit monochrome data can be used directly for analysis purpose, but for demonstrating purpose, the input 12-bit image is converted to 8-bit and displayed.
To use See3CAM_10CUG_C
Sample code by default streams See3CAM_10CUG_C. Register the Bayer Filter provided by e-con in the package. The result image is by default interpolated and then displayed.
/* OpenCV Streaming Sample Custom format - Y16 and BY8 To test the Camera's Set the corresponding defines to 1 (CU40 - RGB IR Camera, CU51_12CUNIR - CU51/ 12CUNIR camera) For 10CUG make both defines as 0 */ #include "opencv2/imgproc/imgproc.hpp" #include "opencv2/highgui/highgui.hpp" #include <iostream> #include <conio.h> using namespace cv; using namespace std; #define CU40 0 #define CU51_12CUNIR 0 //Both CU51 or 12CUNIR #define ImageWidth 1280 #define ImageHeight 720 // Actual Data format BGIRR after conversion BGGR - IR is replaced with the G //IR data is collected as a separate image bool ConvertRGIR2RGGB(Mat BayerRGIR, Mat &BayerRGGB, Mat &IRimage) { //Result image after replacing the IR pixel with the G data BayerRGGB = BayerRGIR.clone(); //IR data will be half the size of Bayer Image IRimage = Mat(BayerRGIR.size().height / 2, BayerRGIR.size().width / 2, CV_8UC1); //copying the IR data and replacing the IR data with G for (int Row = 0; Row < BayerRGIR.rows; Row += 2) { for (int Col = 0; Col < BayerRGIR.cols; Col += 2) { //Set the IR Data with Nearby Green BayerRGGB.at<uchar>(Row + 1, Col) = BayerRGIR.at<uchar>(Row, Col + 1); //Set the IR Data IRimage.at<uchar>(Row / 2, Col / 2) = BayerRGIR.at<uchar>(Row + 1, Col); } } return true; } // Main Function int main() { cout << "e-con's Sample OpenCV Application to Custom Formats" << endl; cout << endl <<"Demonstrates the working of e-con's Custom Format(Y16 / BY8)"; cout << "cameras with the modified libraries of OpenCV" << endl << endl; char keyPressed; VideoCapture _CameraDevice; Mat ResultImage, InputImage; Mat BayerFrame8, IRImage, BGRImage; //Open the device at the ID 0 _CameraDevice.open(0); if( !_CameraDevice.isOpened()) //Check for the device { cout << endl << "\tCamera Device not Initialised Successfully" << endl << endl; cout << endl << "Press any Key to exit the application" << endl << endl; _getch(); return 0; } //Set up the width and height of the camera _CameraDevice.set(CV_CAP_PROP_FRAME_WIDTH, ImageWidth); _CameraDevice.set(CV_CAP_PROP_FRAME_HEIGHT, ImageHeight); cout << endl << "Press 'Q / q /Esc' key on the image winodw to exit the application" << endl << endl; while(1) { _CameraDevice >> InputImage; //Read the input image if(InputImage.empty()) //Check for the vlid image { cout << "No frame grabbed!!, check whether the camera is free!!" << endl << endl; break; } #if CU51_12CUNIR //Convert to 8 Bit: //Scale the 12 Bit (4096) Pixels into 8 Bit(255) (255/4096)= 0.06226 convertScaleAbs(InputImage, ResultImage, 0.06226); namedWindow("Y16 to Y8", WINDOW_AUTOSIZE); imshow("Y16 to Y8", ResultImage); #elif CU40 //Convert to 8 Bit: //Scale the 10 Bit (1024) Pixels into 8 Bit(255) (255/1024)= 0.249023 convertScaleAbs(InputImage, BayerFrame8, 0.249023); //Filling the missing G -channel bayer data ConvertRGIR2RGGB(BayerFrame8, BayerFrame8, IRImage); //Actual Bayer format BG but Opencv uses BGR & Not RGB So taking RG Bayer format cvtColor(BayerFrame8, BGRImage, COLOR_BayerRG2BGR); namedWindow("Camera BGR Frame", WINDOW_AUTOSIZE); imshow("Camera BGR Frame", BGRImage); namedWindow("Camera IR Frame", WINDOW_AUTOSIZE); imshow("Camera IR Frame", IRImage); #else //10CUG and other camera's namedWindow("Camera Frame", WINDOW_AUTOSIZE); imshow("Camera Frame", InputImage); #endif keyPressed = waitKey(1); //Waits for a user input to quit the application if(keyPressed == 27 || keyPressed == 'q' || keyPressed == 'Q' ) { destroyAllWindows(); break; } } //Release the devices _CameraDevice.release(); cout << endl << "Press any Key to exit the application" << endl << endl; _getch(); return 1; }
With this blog, it is been made clear to access e-con’s custom format Camera’s with OpenCV. The modified OpenCV binary provided for download is built using Visual Studio 2012. For installation instructions of OpenCV with Custom format in Windows / Linux refer to the document provided in the package.
21 Comment
How can I access the format for See3CAM_CU40 ? Thanks a lot!
Hi, our company bought your See3CAM_130 cameras and tested your cameras with e-cam view program. The test result was good for our use especially for aufocus function with an ROI-setting function. Problem is we have to use a separate program in which a 13 meagapixel image is captured from your cameras and the captured image should be processed to obtain further information from the image. We use Windows 7 OS and C++ language under visual studio 2010. Can you provide us a sample code for this use?
See3CAM_130 is a UVC camera and can be accessed using directshow or any open source platforms like OpenCV etc.
Hi,
I’m using the See3CAM_CU40 camera. I managed to get the two images using the code from above with OpenCV 2.4.9 and Visual Studio 2013. Can you provide the OpenCV binary for debug mode?
Thank you!
We don’t have the Open binary for debug mode. If you want Open binary for debug mode you have to create by your self with help of our document given in our blog. For further information, please contact camerasolutions@e-consystems.com
Hello I am trying to use the See3CAM_CU40 with cv2 in python inside of a flask web app but I haven’t been able to get it to work. It either crashes, returns an array of all zeros that can’t be further processed by cv2, or returns none-type. What do I need to do to get it working in python either through cv2.VideoCapture() or a direct serial communication?
Please create the support ticket for your queries. One of our technical expert engineer will discuss further with you over the ticket thread.
https://www.e-consystems.com/create-ticket.asp
I would like to access data from the camera when a new frame is available, based on the frame rate set, instead of grabbing the data from the camera, as the data that I grabbed could be the previous frame, or several frames from the previous frame. I would like to grab each new frame from the camera. Is it possible?
May I know what is the readout time for 1 line of the CMOS camera?
Could you please confirm whether you would like to decide when you have to capture the images instead of continuous capturing? If yes, then this is nothing but triggering. There are triggering options available with couple of our cameras. Please confirm if this is your requirement. Also the readout time varies based on cameras. So in which camera you want the readout time .
Is it possible to access 1080p video stream with OpenCV?
Yes, it’s possible to access 1080p video stream with OpenCV.
Hi, i buy the CU40, and i want c++ with win10.
I using this code, but he say, ‘No frame grabbed!!, check theter the camera is free!!’.
I don’t know this Problem.
We will create the support ticket for your issue. One of our technical expert engineers will discuss further with you over the ticket thread.
Hi,
I’ve made some adjustments to your OpenCV patch to include support for cameras which optionally support Y16 (currently setting fourcc codes with the OpenCV DShow backend is basically broken).
I would like to submit an official pull request, because this will benefit a lot of machine vision users. Can I include your patch code in the pull request? (Otherwise I’m happy to re-implement it, but the solution will be largely similar!).
Best,
Josh
Please proceed as you requested.
I am planning to buy this camera, but I would like to check if the provided code work with the latest version of OpenCV (>3) and Visual Studio (2015)
We have developed sample aplication in OpenCV with C++ and tested our USB camera in visual studio 2012, 2013, 2015. In all these versions we are able to compile the code. If you have followed the document correctly then you will not get any error. If you face any issue in the future, feel free to reach us at techsupport@e-consystems.com. We will assists you in all the best possible ways.
How about python API?
Does this code when converted to python still valid?
We haven’t tested our camera in OpenCV python. But there is a similar API available for python.
Please refer the below link and port it your code
https://gist.github.com/bhive01/9a06c0736e6a416af6eebac3f5026728
Hi, I am using see3CAM_CU55_MH and trying to capture a 12bit image using Opencv (using Python 3.4). But I am getting 8bit images only.
“set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter.fourcc(‘Y’,’1′,’2′,’ ‘))” I’m using this code for conversion. Is there any other way i can get 12bit images using Opencv and python 3.4
We are having an option to take Y12 frames using the OpenCV. For Python, it was in development.
Please refer the below link: https://github.com/econsystems/opencv