python - filling gaps on an image using numpy and scipy

Question

The image (test.tif) is attached. The np.nan values are the whitest region. How to fill those whitest region using some gap filling algorithms that uses values from the neighbours?

import scipy.ndimage

data = ndimage.imread('test.tif')

Answer 1

As others have suggested, scipy.interpolate can be used. However, it requires fairly extensive index manipulation to get this to work.

Complete example:

from pylab import *
import numpy
import scipy.ndimage
import scipy.interpolate
import pdb

data = scipy.ndimage.imread('data.png')

# a boolean array of (width, height) which False where there are missing values and True where there are valid (non-missing) values
mask = ~( (data[:,:,0] == 255) & (data[:,:,1] == 255) & (data[:,:,2] == 255) )

# array of (number of points, 2) containing the x,y coordinates of the valid values only
xx, yy = numpy.meshgrid(numpy.arange(data.shape[1]), numpy.arange(data.shape[0]))
xym = numpy.vstack( (numpy.ravel(xx[mask]), numpy.ravel(yy[mask])) ).T

# the valid values in the first, second, third color channel,  as 1D arrays (in the same order as their coordinates in xym)
data0 = numpy.ravel( data[:,:,0][mask] )
data1 = numpy.ravel( data[:,:,1][mask] )
data2 = numpy.ravel( data[:,:,2][mask] )

# three separate interpolators for the separate color channels
interp0 = scipy.interpolate.NearestNDInterpolator( xym, data0 )
interp1 = scipy.interpolate.NearestNDInterpolator( xym, data1 )
interp2 = scipy.interpolate.NearestNDInterpolator( xym, data2 )

# interpolate the whole image, one color channel at a time    
result0 = interp0(numpy.ravel(xx), numpy.ravel(yy)).reshape( xx.shape )
result1 = interp1(numpy.ravel(xx), numpy.ravel(yy)).reshape( xx.shape )
result2 = interp2(numpy.ravel(xx), numpy.ravel(yy)).reshape( xx.shape )

# combine them into an output image
result = numpy.dstack( (result0, result1, result2) )

imshow(result)
show()

Output:

This passes to the interpolator all values we have, not just the ones next to the missing values (which may be somewhat inefficient). It also interpolates every point in the output, not just the missing values (which is extremely inefficient). A better way is to interpolate just the missing values, and then patch them into the original image. This is just a quick working example to get started :)