python - Why doesn't first and last in a groupby give me first and last

Question

I'm posting this because the topic just got brought up in another question/answer and the behavior isn't very well documented.

Consider the dataframe df

df = pd.DataFrame(dict(
    A=list('xxxyyy'),
    B=[np.nan, 1, 2, 3, 4, np.nan]
))

   A    B
0  x  NaN
1  x  1.0
2  x  2.0
3  y  3.0
4  y  4.0
5  y  NaN

I wanted to get the first and last rows of each group defined by column 'A'.

I tried

df.groupby('A').B.agg(['first', 'last'])

   first  last
A             
x    1.0   2.0
y    3.0   4.0

However, This doesn't give me the np.NaNs that I expected.

How do I get the actual first and last values in each group?

Answer 1

As noted here by @unutbu:

The groupby.first and groupby.last methods return the first and last non-null values respectively.

To get the actual first and last values, do:

def h(x):
    return x.values[0]

def t(x):
    return x.values[-1]

df.groupby('A').B.agg([h, t])

     h    t
A          
x  NaN  2.0
y  3.0  NaN