Many developers will never encounter issues when floating-point math fails to return the correct result. Every once in a while, when you’re expecting to see a number like 10.1 in your trace panel, you might actually see 10.1000000001. This isn’t a specific issue with Flash either. By it’s very nature, any sort of calculation with floating-point numbers can result in tiny, nearly insignificant errors anywhere in modern computing.
In many cases, that number could get corrected the next time you use it in a mathematical operation. Other times, you’ll never notice that something went wrong because you might be animating some object, and the position gets rounded off anyway. You won’t always be so lucky, though. At some point in your future, you may need to display a number to the user. Imagine their look of shock as some crazy number with ten decimal places appears on the screen. “Oh my god! Why is it doing that?” exclaims a user of your cool new Flex app.
It gets worse. When these errors occur, checking if two numbers are equal can fail as well. Now, when you need to know if your function returned a multiple of two, and you get 15.9999999999, bad things can happen. Another problem, which I’ve run into a few times, is when I’m “counting” between two numbers, like two and three:
2 2.1 2.2 2.3 2.4 2.5000000001 2.6 2.7000000001 2.8 2.9 3
Consider the following two functions. I’ve been using them for a while now, and they work great. The first, correctFloatingPointError will manipulate the number (even a fractional one) with some clever multiplication, division, and rounding to try to account for the error. The second fuzzyEquals will check if two numbers are almost equal. In many cases, if two numbers have a difference of 0.00001, they might as well be considered equal. With both of these functions, you can manipulate the precision to allow for more accurate or inaccurate calculations depending on your needs.
/*
Copyright (c) 2007 Josh Tynjala
Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation
files (the "Software"), to deal in the Software without
restriction, including without limitation the rights to use,
copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.
*/
/**
* Corrects errors caused by floating point math.
*/
public function correctFloatingPointError(number:Number, precision:int = 5):Number
{
//default returns (10000 * number) / 10000
//should correct very small floating point errors
var correction:Number = Math.pow(10, precision);
return Math.round(correction * number) / correction;
}
/**
* Tests if two numbers are almost equal.
*/
public function fuzzyEquals(number1:Number, number2:Number, precision:int = 5):Boolean
{
var difference:Number = number1 - number2;
var range:Number = Math.pow(10, -precision);
//default check:
//0.00001 < difference > -0.00001
return difference < range && difference > -range;
}
If you have any more tips in this area, I’d like to hear them. Those two functions have been enough for my needs so far, but my toolbox always has room for an extra trick or two.
WTF, I can imagine when perfoming a comparison this could be a nightmare especially if the next time you access it it corrects itself. Havent seen it yet but thanks for the heads up
)
Like I said, it should rarely affect most developers. Unfortunately, I’m working on a project that displays many numbers, and I need them to stay as accurate as possible.
Here’s the official word from Adobe in TechNote aimed at Flash users, but also includes examples in JavaScript, VBScript, and Perl to help indicate that Flash Player is not the cause.
WTF x 2!! Josh, I’ve never run into this issue before, but it’s enlightening! I’ve added this post to my del.icio.us bookmarks, as I’m sure it will happen someday (just my luck!).
Thanks for the heads-up!
It happens the other way around too.
Try this in a blank flash file:
trace(10.00000000000001);//outputs 10
trace(10.0000000000001);//outputs 10.0000000000001
cheers
Awesome, thanks. I am doing a financial services app in Flex and I can’t just cavalierly wave my hand and say “it’s just the nature of computing” like Adobe does. I was getting something as basic as 0.08 * 0.7 = 0.55999999999999994. This function will come in very handy, thank you!
[...] there are a few adequate work-arounds (please note that these following two methods were based on a similar article from ZeusLabs for inclusion in the SASAPI Framework: /** * Correct "roundoff errors" in floating point [...]
[...] Math.round, or roundToPrecision late as possible, and only when you have [...]
Good post, Just covered some other strategies when working with floating points here: http://troyworks.com/blog/?p=70, and here’s a related function for you collection:
public function notClose(number1:Number, number2:Number, precision:int = 0):Boolean { var difference:Number = number1 - number2; var range:Number = Math.pow(10, -precision); trace(-range + " " + difference + " " + range); var res:Boolean = ( difference < -range ) || ( range > difference ) ; trace("notClose" + res); return res; }Josh, thanks a lot man! I actually did run into this while trying to do some AS based audio fades. Your “correctFloatingPointError” function smoothed that out nicely.
[...] I originally found this information/code on a post called Floating-point errors got you down? [...]
[...] that would have involved many program operations to simulate each BCD calculation. Â Then this blog by Josh Tynjala gave me another idea. Â (Ok, probably not a new idea – but it was a revelation to [...]
[...] Floating-point errors got you down? – Josh Talks Flash Many developers will never encounter issues when floating-point math fails to return the correct result. Every once in a while, when you’re expecting to see a number like 10.1 in your trace panel, you might actually see 10.1000000001. This isn’t a specific issue with Flash either. By it’s very nature, any sort of calculation with floating-point numbers can result in tiny, nearly insignificant errors anywhere in modern computing. [...]
Thans man that really worked like magic! dont know what these flex guys doing