python - Scipy: lognormal fitting

Question

There have been quite a few posts on handling the lognorm distribution with Scipy but i still dont get the hang of it.

The 2 parameter lognormal is usually described by the parameters muand sigma which corresponds to Scipys loc=0 and sigma=shape, mu=np.log(scale).

At scipy, lognormal distribution - parameters, we can read how to generate a lognorm(mu,sigma)sample using the exponential of a random distribution. Now lets try something else:

A)

Whats the problem in creating a lognorm directly:

# lognorm(mu=10,sigma=3)
# so shape=3, loc=0, scale=np.exp(10) ?
x=np.linspace(0.01,20,200)
sample_dist = sp.stats.lognorm.pdf(x, 3, loc=0, scale=np.exp(10))
shape, loc, scale = sp.stats.lognorm.fit(sample_dist, floc=0)
print shape, loc, scale
print np.log(scale), shape # mu and sigma
# last line: -7.63285693379 0.140259699945  # not 10 and 3

B)

I use the return values of a fit to create a fitted distribution. But again im doing something wrong apparently:

samp=sp.stats.lognorm(0.5,loc=0,scale=1).rvs(size=2000) # sample
param=sp.stats.lognorm.fit(samp) # fit the sample data
print param # does not coincide  with shape, loc, scale above!
x=np.linspace(0,4,100)
pdf_fitted = sp.stats.lognorm.pdf(x, param[0], loc=param[1], scale=param[2]) # fitted distribution
pdf = sp.stats.lognorm.pdf(x, 0.5, loc=0, scale=1) # original distribution
plt.plot(x,pdf_fitted,'r-',x,pdf,'g-')
plt.hist(samp,bins=30,normed=True,alpha=.3)

Answer 1

I made the same observations: a free fit of all parameters fails most of the time. You can help by providing a better initial guess, fixing the parameter is not necessary.

samp = stats.lognorm(0.5,loc=0,scale=1).rvs(size=2000)

# this is where the fit gets it initial guess from
print stats.lognorm._fitstart(samp)

(1.0, 0.66628696413404565, 0.28031095750445462)

print stats.lognorm.fit(samp)
# note that the fit failed completely as the parameters did not change at all

(1.0, 0.66628696413404565, 0.28031095750445462)

# fit again with a better initial guess for loc
print stats.lognorm.fit(samp, loc=0)

(0.50146296628099118, 0.0011019321419653122, 0.99361128537912125)

You can also make up your own function to calculate the initial guess, e.g.:

def your_func(sample):
    # do some magic here
    return guess

stats.lognorm._fitstart = your_func