python - How do you handle column names having spaces in them when using pd.read_clipboard?

Question

This is a real problem I've faced for a long time.

Take this dataframe:

         A         B  THRESHOLD
       NaN       NaN        NaN
 -0.041158 -0.161571   0.329038
  0.238156  0.525878   0.110370
  0.606738  0.854177  -0.095147
  0.200166  0.385453   0.166235

It is easy enough to copy using pd.read_clipboard. However, if one of the column names has a space:

         A         B     Col #3
       NaN       NaN        NaN
 -0.041158 -0.161571   0.329038
  0.238156  0.525878   0.110370
  0.606738  0.854177  -0.095147
  0.200166  0.385453   0.166235

Then, it is read like this:

          A         B       Col  #3
0       NaN       NaN       NaN NaN
1 -0.041158 -0.161571  0.329038 NaN
2  0.238156  0.525878  0.110370 NaN
3  0.606738  0.854177 -0.095147 NaN
4  0.200166  0.385453  0.166235 NaN

How can I prevent that?

Answer 1

What I do in this situation is that I make all my columns two or more spaces apart, then I use sep='ss+' for my delimiter, this way when I do have column headings with a single space such as, Col #3 above it treats it as one column.

         A         B     Col #3
       NaN       NaN        NaN
 -0.041158  -0.161571   0.329038
  0.238156   0.525878   0.110370
  0.606738   0.854177  -0.095147
  0.200166   0.385453   0.166235

df = pd.read_clipboard(sep='ss+')

You do get this warning, but you can ignore it since it as done it right. Or you could put the engine='python' if your OCD gets the best of you. :)

C:Program FilesAnaconda3libsite-packagespandasioclipboards.py:63: ParserWarning: Falling back to the 'python' engine because the 'c' engine does not support regex separators (separators > 1 char and different from 's+' are interpreted as regex); you can avoid this warning by specifying engine='python'. return read_table(StringIO(text), sep=sep, **kwargs)

print(df)

          A         B    Col #3
0       NaN       NaN       NaN
1 -0.041158 -0.161571  0.329038
2  0.238156  0.525878  0.110370
3  0.606738  0.854177 -0.095147
4  0.200166  0.385453  0.166235