opencv,image-processing,3d,camera-calibration , Finding 3D coordinate of object

Finding 3D coordinate of object


Tag: opencv,image-processing,3d,camera-calibration

I try to find 3D coordinate (in world )of abject that I know his coordinate in image .so after some research in internet I succed to find X and Y cordinate. but I can't find Z

that is my program in openCV

  void drawAndCalcul3D(int x,int y,Mat& frame)
cv::Mat uvPoint = cv::Mat::ones(3,1,cv::DataType<double>::type);<double>(0,0) = x; //got this point using mouse callback<double>(1,0) = y;

std::vector<cv::Point2f> imagePoints;
std::vector<cv::Point3f> objectPoints;
cv::Mat rotationMatrix(3,3,cv::DataType<double>::type);
Mat cameraMatrix(3,3,DataType<double>::type);
  setIdentity(cameraMatrix);<double>(0,0)=493.415; //my Camera matrix<double>(0,1)=0;<double>(0,2)=319.5;<double>(1,0)=0;<double>(1,1)=493.415;<double>(1,2)=179.5;<double>(2,0)=0;<double>(2,1)=0;<double>(2,2)=1;
// extrinsic parameters rvec and tvect : rotation and translation matrix
Mat rvec(3,1,cv::DataType<double>::type);<double>(0)=-0.1408;<double>(1)=3.011;<double>(2)=-0.171147;
cv::Mat tvec(3,1,cv::DataType<double>::type);<double>(0)=15.84;<double>(1)=-64.67;<double>(2)=274.584;

cv::Mat tempMat, tempMat2;
double s;
tempMat = rotationMatrix.inv() * cameraMatrix.inv() * uvPoint;
tempMat2 = rotationMatrix.inv() * tvec;
s<double>(2,0); //12 represents the height Zconst
s /=<double>(2,0);
Mat exp=rotationMatrix.inv() * (s * cameraMatrix.inv() * uvPoint - tvec);
cv::putText(frame,intToString(<double>(0))+ " , " + intToString(<double>(1))+ " , " + intToString(<double>(2)),cv::Point(x,y+20),2,1,Scalar(0,255,0));

can any one know how can I find the depth of object (Z) ?

I use the same program her Computing x,y coordinate (3D) from image point I don't know how I can change it to find Z when it's not constant


x, y image point only determines a ray from the camera centers through the image point. It has infinite number of possible z and when you multiply images point with inverse matrices you will get an equation of a ray or a line. It is impossible to get 3D from a single image point using a single camera. Unless of course you make some strong assumptions such as object shape or size or use extra knowledge such as monocular cues. Examples of such cues are blur/focus, brightness/shading, texture size, familiar size of other objects nearby, etc.

Another example - if you know an equation of the plane where the object lies you can intersect this plane with the ray and arrive at a unique 3D point. Still another example, say, you use an accelerometer in a cell phone which gives you an angle where camera looks relative to the gravity vector. You also know the approximate height at which you hold a cell phone (1.5m above the ground). Than you can easily calculates 3D points on the ground as intersection of your rays with known plane, see below. The formula gives Z for the point in the center of the image. For other points you have to add extra angle formed by off-center pixels. This will allow you to completely restore 3D of the ground plane.

enter image description here


OpenCV - Method 'knnMatch' could not be resolved

I have just a little problem with the right resource. I am using opencv 2.4.8 and I couldn't find the right resource for knnMatch(). I tried the following which didn't work: #include "opencv2/core/core.hpp" #include <opencv2/nonfree/features2d.hpp> #include "opencv2/nonfree/nonfree.hpp" #include <opencv2/ml/ml.hpp> //#include "opencv2/features2d/features2d.hpp" //#include <opencv2/legacy/legacy.hpp> Can someone say me the right resource?...

Zero out portion of multidim numpy array

I have an numpy array with dimensions (200, 200, 3). It is an RGB image. I also have the (xmin,ymin,xmax,ymax) coordinates of a region of this image that I would like to set to zero. This region should be zero in all three channels. I can of course solve this...

opencv convertTo not working

I know this question has been asked and answered by others. But I still can't not solve my question. I read a frame from a video, which has format unsigned char (CV_8U). I hope to convert it to double precision(CV_64F). I do as following: VideoCapture capture(fileName); Mat image; capture >>...

Bandpass Filter in Python for Image Processing

I have a noisy dataset (a stack of images) which films dim particles moving about some really bright artefacts (which are immobilized). I would like to somehow remove the immobilized artefacts from the images by applying some sort of bandpass filter wherein only pixels within a specific range are converted...

opencv window not refreshing at mouse callback

I am trying to draw with mouse move in an opencv window. But when I draw, nothing draws on the window. When I try to close the window from the cross in the topleft(ubuntu), it opens a new window which it should be as I haven't pressed escape, and in...

How to call OpenCV's MatchTemplate method from C#

I'm trying to get the template's position on the image using OpenCVSharp library from NuGet. Here is the code I've wrote: var image = Cv.LoadImage("Image.png"); var template = Cv.LoadImage("Template.png"); var w = (image.Width - template.Width) + 1; var h = (image.Height - template.Height) + 1; IplImage result = new IplImage(w,...

Sending live video frame over network in python opencv

I'm trying to send live video frame that I catch with my camera to a server and process them. I'm usig opencv for image processing and python for the language. Here is my code import cv2 import numpy as np import socket import sys import pickle cap=cv2.VideoCapture(0) clientsocket=socket.socket(socket.AF_INET,socket.SOCK_STREAM) clientsocket.connect(('localhost',8089))...

Using OpenCV in Swift iOS

After adding the OpenCV 2 framework in my xcode project, I tried searching for samlpes or tutorials for integration with swift. Are there any good tutorials for the same?...

Sending a Mat object over socket from Java to Java

I understand Sockets over Java and sending Int,String,bytes etc over it. What i just want to know is that is there a way to decode a Mat object to byte array and then send it over java socket and then retrieve back the Mat object from the byte received? Till...

Create mask from bwtraceboundary in Matlab

I'm trying to create a mask (or similar result) in order to erase pieces of a binary image that are not attached to the object surrounded by the boundary. I saw this thread ( to do this from bwboundaries, but I'm having trouble making suitable changes to it. My goal...

Camera calibration and conversion of coordinates(OpenCV)

I am trying to map the 2D pixel coordinates in an image to real world 3D coordinates with respect to a fixed webcam. The calibration tutorial on the OpenCV page ( has given me the following xml file: <?xml version="1.0"?> <opencv_storage> <calibration_Time>"Wed Jun 17 12:02:01 2015"</calibration_Time> <nrOfFrames>25</nrOfFrames> <image_Width>640</image_Width> <image_Height>480</image_Height>...

c++ read in image set with different file names without hardcoding

Is there any way of reading in a set of images from file that all have varying names to each other, i.e. no continuity at all? So if you had 4 images in the same folder with the file names of: head.jpg shoulders.png knees.tiff toes.bmp Without hard coding the file...

Error for cv::FileStorage in JNI

I am getting errors while compiling my native.cpp file in Eclipse ADT/NDK. While compiling the following lines of code FileStorage storage(nativepath, FileStorage::WRITE); storage << "img" << mat; storage.release(); The errors are D:/androidworkspace/Augmented-Reality//obj/local/arm64-v8a/objs/ndksetup/native.o: In function Java_com_shahrukh_AugmentedReality_CAMShiftDetection_savemat(_JNIEnv*, _jobject*, long long, _jstring*)': D:\androidworkspace\Augmented-Reality/jni/native.cpp:14: undefined reference...

Reading all the files in sequence in MATLAB

I am trying to read all the images in the folder in MATLAB using this code flst=dir(str_Expfold); But it shows me output like this. which is not the sequence as i want. Can anyone please tell me how can i read all of them in sequence? for giving downmark, please...

Must compile Opencv with Mingw in order to use in QT under Winodws?

I've visit these blogs How to link opencv in QtCreator and use Qt library All of them are using Mingw to compile Opencv through Cmake. If I want to use Opencv in QT, is compiling with Mingw the only way? I have this question because I already compiled...

Creating a 25fps slow motion video from a 100fps GoPro .mp4 video with C++/OpenCV

I have a 100fps .mp4 GoPro video and I want to create from it a slow Motion one with 25fps. I'm trying since two days but to no avail. I could play the video, save a video from the GoPro's WiFi stream, but when I try to read the 100fps...

OpenCV FAST corner detection SSE implementation walkthrough

Could someone help me understanding the SSE implementation of the FAST corner detection in OpenCV? I understand the algorithm but not the implementation. Could somebody walk me through the code? The code is long, so thank you in advance. I am using OpenCV 2.4.11 and the code goes like this:...

Dividing main function into other functions in opencv using c++

I am learning opencv using c++. As I don't have any background knowledge in c++.I am learning it parallel with opencv. Here is my doubt.My main program is very big.So,I want to divide it into small functions and call them whenever necessary in a loop using conditional statements.I have searched...

Detecting face using haar-like cascade in opencv using c++

I have been trying to execute the below code but couldn't compile it.I searched for error by masking some lines as comments.Finally I reached CascadeClassifier face_cascade;by removing all the other lines from face_cascade.load to rectangle(image,faces[i],Scalar(0,125,165),2,8,0); But I couldn't understand what the error is!Did I declare CascadeClassiier in a wrong way?...

OpenCV Save a Mat as Binary (1-bit depth) TIFF

Suppose we have a Mat after applying the OpenCv's Imgproc.adaptiveThreshold: Mat srcImage = ...; Mat binaryImage = new Mat(); Imgproc.adaptiveThreshold(srcImage, binaryImage, 255, Imgproc.ADAPTIVE_THRESH_GAUSSIAN_C, Imgproc.THRESH_BINARY, THRESHOLD_BLOCK_SIZE, 10); When I save the binaryImage using Highgui.imwrite: boolean isOk = Highgui.imwrite("sample.tiff", binaryImage); Everything is ok except that output TIFF is not actually a binary...

error using already compiled version of openCV

I've used the already compiled version of openCV for Raspberry Pi. link for anyone who is interested After trying to compile using this command line g++ test3.cpp -o test3 -I/usr/local/include/ -lraspicam -lraspicam_cv -L/opt/vc/lib -lmmal -lmmal_core -lmmal_util -I/usr/include -lopencv_core -lopencv_highgui -lopencv_imgproc -lwiringPi -lpthread I get the following error lines. //usr/local/lib/ undefined...

Saving images with more than 8 bits per pixel in matlab

I need to save a set of pre-processing images in matlab, resulting in grayscale images. The problem is the fact that these pre-processed images have pixel values higher than 255. If I save them with imwrite() as, for instance, .PNG files, does matlab normalize the values to be in [0,255]...

Connecting two binary objects in matlab

I have a binary matrix containing several binary objects and I want to bridge between them. Actually I have the following picture: And the result has to be like this: Is there any function or a shortcut way, other than loops, for this problem?...

best way to create a mat from a CIImage?

I am using CIDetector to detect faces, then using OpenCV on the lower half of each face to detect the size of any smiles. I am using the below code to create the cv::mat which OpenCV can perform the detection on, as you can see the image goes through the...

Increase Size of Black Bullets in Image in C#

Given the following image: I need to output this image: I want to increase the size of Black Pixels in the image using C# or AForge.Net...

adaptive thresholding ---ValueError: too many values to unpack

I'm pretty amateur at image processing. I could successfully do normal thresholding but however I'm facing an error in Adaptive Thresholding. Here is my code: import cv2 import numpy as np img = cv2.imread("vehicle004.jpg") img = cv2.medianBlur(img,5) gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY) _,th2=cv2.adaptiveThreshold(gray,255,cv2.ADAPTIVE_THRESH_MEAN_C,cv2.THRESH_BINARY,11,2) cv2.imshow("window2",th2) cv2.waitKey(0) cv2.destroyAllWindows() Error Message: line 7, in <module>...

imread function doesn't work in matlab

I am new to Image processing and learning matlab. Actually till now I have used matlab cloud version where uploading an image was directly possible and didn't face any issue. But now I am facing an extreme problem of uploading an image in matlab offline version software. I imported an...

Visual Studio 2013 LINK : fatal error LNK1181: cannot open input file

I am using Visual Studio 2013. I'm trying to build some code given to me from my professor and I keep getting this error: LINK : fatal error LNK1181: cannot open input file 'C:\Users\manduchi\Documents\eyegaze\EyeGazeDemo..\Libraries\OpenCV\lib\opencv_core249.lib' However, on my computer opencv_core249.lib is located somewhere else. I've tried updating the linker directories to...

Captcha with merging a number with real image

I have searched for captcha generation tools, but I could not find any tool which can generate the below captcha pictures. You can seen these when you want to register on google. It uses a number merged on a real picture and produce a new real image picture. Do you...

Getting an image to render with php

To keep it simple, I'm trying to display an image using php to help try and prevent any kind of a security loophole. I have the image being read through a php script and I am wanting to render it, so instead of going to I want to have...

OpenCV - Detection of moving object C++

I am working on Traffic Surveillance System an OpenCv project, I need to detect moving cars and people. I am using background subtraction method to detect moving objects and thus drawing counters. I have a problem : When two car are moving on road closely them my system detects it...

thicken an object of image to a curve in matlab

I have a labeled matrix containing two objects. How can I thicken an object to a curve? Actually I have the following image: and I want this: Each pixel of the resulting curve is the median of each column. But if you have another idea, it is acceptable, because I...

How can I replace the white rectangle within an image using ImageMagick?

Overview: The first picture is my original image. Here I want to replace the white rectangle shown with another image. My approach: I have created a mask image using floodfill and it looks as: Problem: Now I would like to get the distance or co-ordinates of the rectangle in the...

Displaying a 32-bit image with NaN values (ImageJ)

I wrote a multilanguage 3-D image denoising ImageJ plugin that does some operations on an image and returns the denoised image as a 1-D array. The 1-D array contains NaN values (around the edges). The 1-D array is converted back into an image stack and displayed. It is simply black....

Camera Calibration with OpenCV: Using the distortion and rotation-translation matrix

I am reading the following documentation: I have managed to successfully calibrate the camera obtaining the camera matrix and the distortion matrix. I had two sub-questions: 1) How do I use the distortion matrix as I don't know 'r'? 2) For all the views I have the rotation and...

Extracting polygon given coordinates from an image using OpenCV

I've a set of points like the following: <data:polygon> <data:point x="542" y="107"/> <data:point x="562" y="102"/> <data:point x="582" y="110"/> <data:point x="598" y="142"/> <data:point x="600" y="192"/> <data:point x="601" y="225"/> <data:point x="592" y="261"/> <data:point x="572" y="263"/> <data:point x="551" y="245"/> <data:point x="526" y="220"/> <data:point x="520" y="188"/> <data:point x="518" y="152"/> <data:point x="525" y="127"/> <data:point...

Error : Cmake can't generate openCV

I am trying to instal opencv C++ with codeBlocks in Windows 8 by following this but I am blocked in step 4.when I try to generate Cmake I have this Error : CMake Error: CMake was unable to find a build program corresponding to "MinGW Makefiles". CMAKE_MAKE_PROGRAM is not...

how to detect partial corrupted images in Jpg/Jpeg format

In a very big image dataset, we have some corrupted images as following images. These images can be viewed without any problem, but human eyes can see some corrupted areas in gray color. How can I detect these corrupted images´╝č Actually I have written a script in Matlab for detecting....

Extracting Points from Lines using OpenCV

I am trying to extract points from a line on an image using openCV in C++ Language. The line is programmed to display on the image, but I need to know how do you extract points from line and input it into a text file?

How to create thumbnails using opencv-python?

I'm trying to downsample my image(using anti-aliasing) using Python-Pillow's im.thumbnail() method. My code looks like: MAXSIZE = 1024 im.thumbnail(MAXSIZE, Image.ANTIALIAS) Can you tell me some alternative in opencv-python to perform this re-sizing operation ?...

Is there a way to prevent rounding in opencv matrix divison

I have an integer matrix and I want to perform an integer division on it. But opencv always rounds the result. I know I can divide each element manually but I want to know is there a better way for this or not? Mat c = (Mat_ <int> (1,3) <<...

Surface normal on depth image

How to estimate the surface normal of point I(i,j) on a depth image (pixel value in mm) without using Point Cloud Library(PCL)? I've gone through (1), (2), and (3) but I'm looking for a simple estimation of surface normal on each pixel with C++ standard library or openCV. ...

Java encode raw bytes into image simple image fomat/ file

I am very new to image encoding and would rather not learn a whole lot about it. Basically I'm taking greyscale byte array where each byte equals one pixel. I'm getting this data from mnist where I get 28x28 byte images. Anyway, bellow is my code, so you understand what...

How to prevent exceeding matrix dimensions while dividing an image into blocks?

I have an image which I want to divide into overlapping blocks. I have set the box to be of size 8 rows and 8 columns, and the overlapping factor to be 4 rows/columns. This is what I have written to solve this: img = imread('a03-017-05.png'); overlap = 4 count...