gotchas where Numpy differs from straight python?

Question

Folks,

is there a collection of gotchas where Numpy differs from python, points that have puzzled and cost time ?

"The horror of that moment I shall never never forget !"
"You will, though," the Queen said, "if you don't make a memorandum of it."

For example, NaNs are always trouble, anywhere. If you can explain this without running it, give yourself a point --

from numpy import array, NaN, isnan

pynan = float("nan")
print pynan is pynan, pynan is NaN, NaN is NaN
a = (0, pynan)
print a, a[1] is pynan, any([aa is pynan for aa in a])

a = array(( 0, NaN ))
print a, a[1] is NaN, isnan( a[1] )

(I'm not knocking numpy, lots of good work there, just think a FAQ or Wiki of gotchas would be useful.)

Edit: I was hoping to collect half a dozen gotchas (surprises for people learning Numpy).
Then, if there are common gotchas or, better, common explanations, we could talk about adding them to a community Wiki (where ?) It doesn't look like we have enough so far.

Answer 1

Because __eq__ does not return a bool, using numpy arrays in any kind of containers prevents equality testing without a container-specific work around.

Example:

>>> import numpy
>>> a = numpy.array(range(3))
>>> b = numpy.array(range(3))
>>> a == b
array([ True,  True,  True], dtype=bool)
>>> x = (a, 'banana')
>>> y = (b, 'banana')
>>> x == y
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

This is a horrible problem. For example, you cannot write unittests for containers which use TestCase.assertEqual() and must instead write custom comparison functions. Suppose we write a work-around function special_eq_for_numpy_and_tuples. Now we can do this in a unittest:

x = (array1, 'deserialized')
y = (array2, 'deserialized')
self.failUnless( special_eq_for_numpy_and_tuples(x, y) )

Now we must do this for every container type we might use to store numpy arrays. Furthermore, __eq__ might return a bool rather than an array of bools:

>>> a = numpy.array(range(3))
>>> b = numpy.array(range(5))
>>> a == b
False

Now each of our container-specific equality comparison functions must also handle that special case.

Maybe we can patch over this wart with a subclass?

>>> class SaneEqualityArray (numpy.ndarray):
...   def __eq__(self, other):
...     return isinstance(other, SaneEqualityArray) and self.shape == other.shape and (numpy.ndarray.__eq__(self, other)).all()
... 
>>> a = SaneEqualityArray( (2, 3) )
>>> a.fill(7)
>>> b = SaneEqualityArray( (2, 3) )
>>> b.fill(7)
>>> a == b
True
>>> x = (a, 'banana')
>>> y = (b, 'banana')
>>> x == y
True
>>> c = SaneEqualityArray( (7, 7) )
>>> c.fill(7)
>>> a == c
False

That seems to do the right thing. The class should also explicitly export elementwise comparison, since that is often useful.