python - Save & Retrieve Numpy Array From String

Question

I would like to convert a multi-dimensional Numpy array into a string and, later, convert that string back into an equivalent Numpy array.

I do not want to save the Numpy array to a file (e.g. via the savetxt and loadtxt interface).

Is this possible?

Answer 1

You could use np.tostring and np.fromstring:

In [138]: x = np.arange(12).reshape(3,4)

In [139]: x.tostring()
Out[139]: 'x00x00x00x00x01x00x00x00x02x00x00x00x03x00x00x00x04x00x00x00x05x00x00x00x06x00x00x00x07x00x00x00x08x00x00x00x00x00x00
x00x00x00x0bx00x00x00'

In [140]: np.fromstring(x.tostring(), dtype=x.dtype).reshape(x.shape)
Out[140]: 
array([[ 0,  1,  2,  3],
       [ 4,  5,  6,  7],
       [ 8,  9, 10, 11]])

Note that the string returned by tostring does not save the dtype nor the shape of the original array. You have to re-supply those yourself.

---

Another option is to use np.save or np.savez or np.savez_compressed to write to a io.BytesIO object (instead of a file):

import numpy as np
import io

x = np.arange(12).reshape(3,4)
output = io.BytesIO()
np.savez(output, x=x)

The string is given by

content = output.getvalue()

And given the string, you can load it back into an array using np.load:

data = np.load(io.BytesIO(content))
x = data['x']

This method stores the dtype and shape as well.

For large arrays, np.savez_compressed will give you the smallest string.

---

Similarly, you could use np.savetxt and np.loadtxt:

import numpy as np
import io

x = np.arange(12).reshape(3,4)
output = io.BytesIO()
np.savetxt(output, x)
content = output.getvalue()
# '0.000000000000000000e+00 1.000000000000000000e+00 2.000000000000000000e+00 3.000000000000000000e+00
4.000000000000000000e+00 5.000000000000000000e+00 6.000000000000000000e+00 7.000000000000000000e+00
8.000000000000000000e+00 9.000000000000000000e+00 1.000000000000000000e+01 1.100000000000000000e+01
'

x = np.loadtxt(io.BytesIO(content))
print(x)

---

Summary:

• tostring gives you the underlying data as a string, with no dtype or shape
• save is like tostring except it also saves dtype and shape (.npy format)
• savez saves the array in npz format (uncompressed)
• savez_compressed saves the array in compressed npz format
• savetxt formats the array in a humanly readable format