python - Find unique columns and column membership

Question

I went through these threads:

• Find unique rows in numpy.array
• Removing duplicates in each row of a numpy array
• Pandas: unique dataframe

and they all discuss several methods for computing the matrix with unique rows and columns.

However, the solutions look a bit convoluted, at least to the untrained eye. Here is for example top solution from the first thread, which (correct me if I am wrong) I believe it is the safest and fastest:

np.unique(a.view(np.dtype((np.void, a.dtype.itemsize*a.shape[1])))).view(a.dtype).reshape(-1, 
a.shape[1])

Either way, the above solution only returns the matrix of unique rows. What I am looking for is something along the original functionality of np.unique

u, indices = np.unique(a, return_inverse=True)

which returns, not only the list of unique entries, but also the membership of each item to each unique entry found, but how can I do this for columns?

Here is an example of what I am looking for:

array([[0, 2, 0, 2, 2, 0, 2, 1, 1, 2],
       [0, 1, 0, 1, 1, 1, 2, 2, 2, 2]])

We would have:

u       = array([0,1,2,3,4])
indices = array([0,1,0,1,1,3,4,4,3])

Where the different values in u represent the set of unique columns in the original array:

0 -> [0,0]
1 -> [2,1]
2 -> [0,1]
3 -> [2,2]
4 -> [1,2]

Answer 1

First lets get the unique indices, to do so we need to start by transposing your array:

>>> a=a.T

Using a modified version of the above to get unique indices.

>>> ua, uind = np.unique(np.ascontiguousarray(a).view(np.dtype((np.void,a.dtype.itemsize * a.shape[1]))),return_inverse=True)

>>> uind
array([0, 3, 0, 3, 3, 1, 4, 2, 2, 4])

#Thanks to @Jamie
>>> ua = ua.view(a.dtype).reshape(ua.shape + (-1,))
>>> ua
array([[0, 0],
       [0, 1],
       [1, 2],
       [2, 1],
       [2, 2]])

For sanity:

>>> np.all(a==ua[uind])
True

To reproduce your chart:

>>> for x in range(ua.shape[0]):
...     print x,'->',ua[x]
...
0 -> [0 0]
1 -> [0 1]
2 -> [1 2]
3 -> [2 1]
4 -> [2 2]

To do exactly what you ask, but will be a bit slower if it has to convert the array:

>>> b=np.asfortranarray(a).view(np.dtype((np.void,a.dtype.itemsize * a.shape[0])))
>>> ua,uind=np.unique(b,return_inverse=True)
>>> uind
array([0, 3, 0, 3, 3, 1, 4, 2, 2, 4])
>>> ua.view(a.dtype).reshape(ua.shape+(-1,),order='F')
array([[0, 0, 1, 2, 2],
       [0, 1, 2, 1, 2]])

#To return this in the previous order.
>>> ua.view(a.dtype).reshape(ua.shape + (-1,))