python - Flatten multiple columns in a dataframe to a single column

Question

I have a dataframe like this:

id    other_id_1    other_id_2    other_id_3
1     100           101           102
2     200           201           202
3     300           301           302

I want this:

id    other_id
1     100
1     101
1     102
2     200
2     201
2     202
3     300
3     301
3     302

I can get my desired output easily like this:

to_keep = {}
for idx in df.index:
    identifier = df.loc[idx]['id']
    to_keep[identifier] = []
    for col in ['other_id_1', 'other_id_2', 'other_id_3']:
        row_val = df.loc[idx][col]
        to_keep[identifier].append(row_val)

Which gives me this:

{1: [100, 101, 102], 2: [200, 201, 202], 3: [300, 301, 302]}

I can easily write that to a file. I am struggling to do this in native pandas, however. I would imagine this seeming transposition would be more straightforward, but am struggling...

Answer 1

Well, if you haven't already, set id as the index:

>>> df
   id  other_id_1  other_id_2  other_id_3
0   1         100         101         102
1   2         200         201         202
2   3         300         301         302
>>> df.set_index('id', inplace=True)
>>> df
    other_id_1  other_id_2  other_id_3
id
1          100         101         102
2          200         201         202
3          300         301         302

Then, you can simply use pd.concat:

>>> df = pd.concat([df[col] for col in df])
>>> df
id
1    100
2    200
3    300
1    101
2    201
3    301
1    102
2    202
3    302
dtype: int64

And if you need the values sorted:

>>> df.sort_values()
id
1    100
1    101
1    102
2    200
2    201
2    202
3    300
3    301
3    302
dtype: int64
>>>