python - Converting pandas dataframe to structured arrays

Question

I have the following pandas dataframe

import pandas as pd
a = [2.5,3.3]
b = [3.6,3.9]
D = {'A': a, 'B': b}

which gives me something like

+---+-----+-----+
|   |  A  |  B  |
+---+-----+-----+
| 0 | 2.5 | 3.3 |
| 1 | 3.6 | 3.9 |
+---+-----+-----+ 

I want to convert this dataframe to a structured array like

data = np.rec.array([
('A', 2.5),
('A', 3.6),
('B', 3.3),
('B', 3.9),
], dtype = [('Type','|U5'),('Value', '<i8')])

I failed to find a way to make this happen since I'm new to pandas. I tried pd.to_records but the index is getting in the way and I cannot find a way around that.

Any help is appreciated. Thanks.

Answer 1

Melt the DataFrame to make A and B (the column index) into a column. To get rid of the numeric index, make this new column the index. Then call to_records():

import pandas as pd
a = [2.5,3.3]
b = [3.6,3.9]
D = {'A': a, 'B': b}
df = pd.DataFrame(D)
result = (pd.melt(df, var_name='Type', value_name='Value')
          .set_index('Type').to_records())
print(repr(result))

yields

rec.array([('A',  2.5), ('A',  3.3), ('B',  3.6), ('B',  3.9)], 
          dtype=[('Type', 'O'), ('Value', '<f8')])

---

This is the key step:

In [167]: df
Out[167]: 
     A    B
0  2.5  3.6
1  3.3  3.9

In [168]: pd.melt(df)
Out[168]: 
  variable  value
0        A    2.5
1        A    3.3
2        B    3.6
3        B    3.9

Once you've melted the DataFrame, to_records (basically) returns the desired result:

In [169]: pd.melt(df).to_records()
Out[169]: 
rec.array([(0, 'A',  2.5), (1, 'A',  3.3), (2, 'B',  3.6), (3, 'B',  3.9)], 
          dtype=[('index', '<i8'), ('variable', 'O'), ('value', '<f8')])