python - inequality comparison of numpy array with nan to a scalar

Question

I am trying to set members of an array that are below a threshold to nan. This is part of a QA/QC process and the incoming data may already have slots that are nan.

So as an example my threshold might be -1000 and hence I would want to set -3000 to nan in the following array

x = np.array([np.nan,1.,2.,-3000.,np.nan,5.])

This following:

x[x < -1000.] = np.nan

produces the correct behavior, but also a RuntimeWarning, but the overhead of disabling the warning

warnings.filterwarnings("ignore")
...
warnints.resetwarnings()

is kind of heavy an potentially a bit unsafe.

Trying to index twice with fancy indexing as follows doesn't produce any effect:

nonan = np.where(~np.isnan(x))[0]
x[nonan][x[nonan] < -1000.] = np.nan

I assume this is because a copy is made due to the integer index or the use of indexing twice.

Does anyone have a relatively simple solution? It would be fine to use a masked array in the process, but the final product has to be an ndarray and I can't introduce new dependencies. Thanks.

Answer 1

One option is to disable the relevant warnings with numpy.errstate:

with numpy.errstate(invalid='ignore'):
    ...

To turn off the relevant warnings globally, use numpy.seterr.