python - Adding error bars to grouped bar plot in pandas

Question

I'm generating a plot in pandas by first generating the following DataFrame:

plotData=resultData.groupby(['student_model','lo_id']).describe().nShots.unstack().reset_index()
plotData['se'] = plotData['std']/np.sqrt(plotData['count'])

The resulting dataframe looks like this:

Then I pivot and plot like so:

plotData.pivot(index='student_model',columns='lo_id',values='mean').plot(kind='bar')

Resulting in the following:

That's all fine, but I need to add the values from the "se" column as errorbars to the plot, and can't get it to work. I know I can add an argument to call to plot (i.e ...plot(kind='bar', yerr=???)), but I don't know how to properly format this to make it work properly. Any ideas?

Answer 1

• Plotting a grouped bar and the corresponding error bars is dependant upon the shape of the dataframe being passed.
• Use .pivot to reshape the dataframe into the correct form to work with yerr.
• It is a key requirement, that when adding yerr as a dataframe, the column headers must be the same as that used for the bars. If the column names are different, the error bars won't show up.
  • An alternate option is to pass an array, as shown in Plotting error bars on grouped bars in pandas
• Tested in python 3.8.11, pandas 1.3.3, matplotlib 3.4.3

import pandas as pd

# dataframe
data = {'class1': ['A', 'A', 'B', 'B'], 'class2': ['R', 'G', 'R', 'G'], 'se': [1, 1, 1, 1], 'val': [1, 2, 3, 4]}
df = pd.DataFrame(data)

  class1 class2  se  val
0      A      R   1    1
1      A      G   1    2
2      B      R   1    3
3      B      G   1    4

# pivot the data
dfp = df.pivot(index='class1', columns='class2', values='val')

class2  G  R
class1      
A       2  1
B       4  3

# pivot the error
yerr = df.pivot(index='class1', columns='class2', values='se')

class2  G  R
class1      
A       1  1
B       1  1

# plot
dfp.plot(kind='bar', yerr=yerr, rot=0)

• Optionally

# or yerr=df.se.reshape((2, 2))
# Where (2, 2) is the shape of df.pivot(index='class1', columns='class2', values='val')
# which is less verbose, but may not be general as generalized