Using gluUnProject to map touches to x,y cords on z=0 plane in Android OpenGL ES 2.0

Question

I've drawn a grid at z=0 in OpenGL 2.0 ES, and just want to convert touch inputs to x/y coordinates on that plane. It seems like this is best done through ray tracing, which involves running gluUnProject on 0 and 1, then creating a ray, solving that ray for z=0?

I found this code, but it is OpenGL ES 1.0: i-schuetz / Android_OpenGL_Picking

Screenshot of app running so you can see camera distortion.

My code on Github, only 4 files. The unproject function I'm trying to write is in MyGLRenderer.java:

    public float[] unproject(float rx, float ry) {
            float rz = 1;
        float[] xyzw = {0, 0, 0, 0};
        int[] viewport = {0, 0, mDisplayWidth, mDisplayHeight};
        android.opengl.GLU.gluUnProject(
                        rx, ry, rz, // window coordinates
                        mViewMatrix, 0,
                        mProjectionMatrix, 0, 
                        viewport, 0,
                        xyzw, 0);
        xyzw[0] /= xyzw[3];
        xyzw[1] /= xyzw[3];
        xyzw[2] /= xyzw[3];
        xyzw[3] = 1;
        return xyzw;
    }

I would like this function to take an rx and ry for the screen, and return an rx and ry for the z=0 plane.

Answer 1

There is nothing particularly special about what gluUnProject (...) does. If you have all of the matrices and the viewport dimensions _{(x,y and width,height)} I can walk you through the process of implementing it yourself.

_{NOTE: I tend to call each coordinate space by a different name than you might be used to, understand that screen space is another name for window space, view space is another name for eye space, object space is another name for model space.}

---

Step 1:

   Screen Space to NDC space _{(Undo Viewport Transform)}

          NDC_X = (2.0 × (Screen_X - Viewport_X) / Viewport_W) - 1.0

          NDC_Y = (2.0 × (Screen_Y - Viewport_Y) / Viewport_H)  - 1.0

   Screen Space to NDC space _{(Undo Depth Range Mapping)}

         Typically in screen space, the Depth Range will map z=0 to near and z=1 to far:

               NDC_Z = 2.0 × Screen_Z - 1.0

---

Step 2^†:

   NDC space to Object space _{(Undo Projection, View and Model Transforms)}

          (Projection Matrix)^-1  × NDC_XYZ1  = View_XYZW

          (ModelView Matrix)^-1 × View_XYZW = Object_XYZW

                  †  _{This can actually be combined into a single step, as you will see below...}

---

                           Object_XYZw = (Projection Matrix × ModelView Matrix)^-1 × NDC_XYZ1

---

Now, you may notice that I crossed-out W in Object_XYZ, we really do not care about this at all but the math will produce a pesky W value nevertheless. At this point, you can return the individual components of Object_XYZ as your r_X, r_Y and r_Z.