Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
949 views
in Technique[技术] by (71.8m points)

floating point - C# float bug? 0.1 - 0.1 = 1.490116E-08

What's going on?! Subtraction works fine until I get to 0.1 - 0.1. I'm in visual c# 2008 using the nonoba.com API.

Console.WriteLine("hit! " + Users[targetNum].character.health + " : " + player.character.profile.attackPower);

Users[targetNum].character.health -= player.character.profile.attackPower;

Console.WriteLine("health! " + Users[targetNum].character.health);

output:

hit! 0.1 : 0.1
health! 1.490116E-08

Thanks all - I might use the decimal type, as I'm normally adding/subtracting nice "round" numbers. For now I'll just go with:

if (Users[targetNum].character.health <= 0.00001)

By the way I knew this wasn't really going to be a "bug" in c# - I thought it would either by a bug in my code or some lack of understanding, which it was.

Having read all the recommended reading, I'm going to conclude that my folly was due to normally using the ActionScript Number type, which maybe has a decimal rather than binary floating point - anyway, it would never give this output.

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

That seems pretty normal for floating point math... you always have to check against a small delta to account for imperceptible rounding differences. Depending on the scenario, decimal might be what you want.

Basically, unless you can be sure that it is exactly the same 0.1 in both cases (i.e. nothing has been done to them), you aren't likely to get zero; in general you'll get something very nearly zero. With decimal you'll usually get more what you expect intuitively.

See also Jon Skeet's pages here:


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...