python - MemoryError when running Numpy Meshgrid

Question

I have 8823 data points with x,y coordinates. I'm trying to follow the answer on how to get a scatter dataset to be represented as a heatmap but when I go through the

X, Y = np.meshgrid(x, y)

instruction with my data arrays I get MemoryError. I am new to numpy and matplotlib and am essentially trying to run this by adapting the examples I can find.

Here's how I built my arrays from a file that has them stored:

XY_File = open ('XY_Output.txt', 'r')
XY = XY_File.readlines()
XY_File.close()

Xf=[]
Yf=[]
for line in XY:
    Xf.append(float(line.split('')[0]))
    Yf.append(float(line.split('')[1]))
x=array(Xf)
y=array(Yf)

Is there a problem with my arrays? This same code worked when put into this example but I'm not too sure.

Why am I getting this MemoryError and how can I fix this?

Answer 1

Your call to meshgrid requires a lot of memory -- it produces two 8823*8823 floating point arrays. Each of them are about 0.6 GB.

But your screen can't show (and your eye can't really process) that much information anyway, so you should probably think of a way to smooth your data to something more reasonable like 1024*1024 before you do this step.