opencv - Does Kinect Infrared View Have an offset with the Kinect Depth View

Question

I am working on a Kinect project using the infrared view and depth view. In the infrared view, using CVBlob library, I am able to extract some 2D points of interest. I want to find the depth of these 2D points. So I thought that I can use the depth view directly, something like this:

coordinates3D[0] = coordinates2D[0];
coordinates3D[1] = coordinates2D[1];
coordinates3D[2] = (USHORT*)(LockedRect.pBits)
[(int)coordinates2D[1] * Width + (int)coordinates2D[0]] >> 3;

I don't think this is the right formula to get the depth. I am able to visualize the 2D points of interest in the depth view. If I get a point (x, y) in infrared view, then I draw it as a red point in the depth view at (x, y)
I noticed that the red points are not where I expect them to be (on an object). There is a systematic error in their locations.

I was of the opinion that the depth view and infrared views have one-to-one correspondence unlike the correspondence between the color view and depth view.
Is this indeed true or is there an offset between the IR and depth views? If there is an offset, can I somehow get the right depth value?

Answer 1

Depth and Color streams are not taken from the same point so they do not correspond to each other perfectly. Also they FOV (field of view) is different.

1. cameras

   • IR/Depth FOV 58.5° x 45.6°
   • Color FOV 62.0° x 48.6°
   • distance between cameras 25mm

2. my corrections for 640x480 resolution for both streams

   if (valid depth)
    {
    ax=(((x+10-xs2)*241)>>8)+xs2;
    ay=(((y+30-ys2)*240)>>8)+ys2;
    }
   

   • x,y are in coordinates in depth image
   • ax,ay are out coordinates in color image
   • xs,ys = 640,480
   • xs2,ys2 = 320,240

   as you can see my kinect has also y-offset which is weird (even bigger then x-offset). My conversion works well on ranges up to 2 m I did not measure it further but it should work even then

3. do not forget to correct space coordinates from depth and depth image coordinates

   pz=0.8+(float(rawdepth-6576)*0.00012115165336374002280501710376283);
   px=-sin(58.5*deg*float(x-xs2)/float(xs))*pz;
   py=+sin(45.6*deg*float(y-ys2)/float(ys))*pz;
   pz=-pz;
   

   • where px,py,pz is point coordinate in [m] in space relative to kinect

   I use coordinate system for camera with opposite Z direction therefore the negation of sign

PS. I have old model 1414 so newer models have probably different calibration parameters