python 3.x - Debugging Numpy VisibleDeprecationWarning (ndarray from ragged nested sequences)

Question

Since NumPy version 19.0, one must specify dtype=object when creating an array from "ragged" sequences. I'm faced with a large number of array calls from my own code and Pandas using threading, line-by-line debugging led me nowhere.

I'd like to figure out which call resulted in VisibleDeprecationWarning in my own code or a call from Pandas. How would I be able to debug this? I've been looking through the source and I cannot see this warning getting called in Python (only in numpy.core._multiarray_umath.cp38-win_amd64.pyd).

Answer 1

With a function that creates a ragged array:

In [60]: def foo(): 
    ...:     print('one') 
    ...:     x = np.array([[1],[1,2]]) 
    ...:     return x 
    ...:                                                                                             
In [61]: foo()                                                                                       
one
/usr/local/bin/ipython3:3: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
  # -*- coding: utf-8 -*-
Out[61]: array([list([1]), list([1, 2])], dtype=object)

I get the warning, but also the expected result.

I can control the warnings.

For example to turn if off:

In [68]: np.warnings.filterwarnings('ignore', category=np.VisibleDeprecationWarning)                 
In [69]: foo()                                                                                       
one
Out[69]: array([list([1]), list([1, 2])], dtype=object)

Or to raise an error:

In [70]: np.warnings.filterwarnings('error', category=np.VisibleDeprecationWarning)                  
In [71]: foo()                                                                                       
one
---------------------------------------------------------------------------
VisibleDeprecationWarning                 Traceback (most recent call last)
<ipython-input-71-c19b6d9633cf> in <module>
----> 1 foo()

<ipython-input-60-6ad21d9e07b4> in foo()
      1 def foo():
      2     print('one')
----> 3     x = np.array([[1],[1,2]])
      4     return x
      5 

VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray

The error gives a traceback telling me where the warning was raised.

There may be ways of refining the warning filter to catch just this one, and not others of the same category. I haven't used this mechanism much.

Read np.warnings.filterwarnings docs for more details.