python - Pandas plot does not overlay

Question

I am trying to overlay a stacked bar chart with a line plot as from the example below but only the second plot is shown and cannot understand why.

import pandas as pd
from matplotlib import pyplot as plt
df=pd.DataFrame({'yarding, mobile cable yarder on trailer': {1928: 1.4027824821879459e-20, 1924: 3.4365045943961052e-37, 1925: 6.9939032596152882e-30, 1926: 1.0712940173393567e-25, 1927: 8.6539917152671678e-23},
                 'yarding and processing, mobile cable yarder on truck': {1928: 1.1679873528237404e-20, 1924: 2.8613089094435456e-37, 1925: 5.8232768671842113e-30, 1926: 8.9198283644271726e-26, 1927: 7.2055027953028907e-23},
                 'delimbing, with excavator-based processor': {1928: 1.6998969986716558e-20, 1924: 4.1643685881703105e-37, 1925: 8.4752370448040848e-30, 1926: 1.2981979323251926e-25, 1927: 1.0486938381883222e-22}})
df2=pd.Series({1928: 3.0638184091973243e-19, 1924: 7.5056562764093482e-36, 1925: 1.5275356821475311e-28, 1926: 2.3398091372066067e-24, 1927: 1.8901157781841223e-21})

ax=df.plot(kind='bar',stacked=True,legend=False)
df2.plot(kind='line',ax=ax)
plt.show()

Answer 1

Line plots plot numerical data against each other.
Bar plots plot numerical data against categorical data. Therefore even if the x values in the bar plot are numbers, the scale on which they are plotted does not correspond to those numbers, but rather to some index.

That means that the x axis scale of a bar plot always goes from 0 to N, where N is the number of bars (Roughly speaking, in reality it's rather -0.5 to N-0.5).

If you now add some values in the range above 1000 to that scale, the bars will shrink until they cannot be seen any more (thus you may think they are not even there).

To circumvent this problem, you may work on two different axes. One for the line plot, one for the bar plot, but have them shared the same y axis.

The following is a possible solution (which is very similar to the solution from Martin, which he has added while I was typing this):

import pandas as pd
from matplotlib import pyplot as plt
df=pd.DataFrame({'yarding, mobile cable yarder on trailer': {1928: 1.4027824821879459e-20, 1924: 3.4365045943961052e-37, 1925: 6.9939032596152882e-30, 1926: 1.0712940173393567e-25, 1927: 8.6539917152671678e-23},
                 'yarding and processing, mobile cable yarder on truck': {1928: 1.1679873528237404e-20, 1924: 2.8613089094435456e-37, 1925: 5.8232768671842113e-30, 1926: 8.9198283644271726e-26, 1927: 7.2055027953028907e-23},
                 'delimbing, with excavator-based processor': {1928: 1.6998969986716558e-20, 1924: 4.1643685881703105e-37, 1925: 8.4752370448040848e-30, 1926: 1.2981979323251926e-25, 1927: 1.0486938381883222e-22}})
df2=pd.Series({1928: 3.0638184091973243e-19, 1924: 7.5056562764093482e-36, 1925: 1.5275356821475311e-28, 1926: 2.3398091372066067e-24, 1927: 1.8901157781841223e-21})

fig, ax = plt.subplots()
# optionally make log scale
ax.set_yscale("log", nonposy='clip')
# create shared y axes
ax2 = ax.twiny()
df.plot(kind='bar',stacked=True,legend=False, ax=ax)
df2.plot(kind='line',ax=ax2)
ax2.xaxis.get_major_formatter().set_useOffset(False)
# remove upper axis ticklabels
ax2.set_xticklabels([])
# set the limits of the upper axis to match the lower axis ones
ax2.set_xlim(1923.5,1928.5)
plt.show()