Many developers will never encounter issues when floating-point math fails to return the correct result. Every once in a while, when you’re expecting to see a number like 10.1 in your trace panel, you might actually see 10.1000000001. This isn’t a specific issue with Flash either. By it’s very nature, any sort of calculation with floating-point numbers can result in tiny, nearly insignificant errors anywhere in modern computing.
In many cases, that number could get corrected the next time you use it in a mathematical operation. Other times, you’ll never notice that something went wrong because you might be animating some object, and the position gets rounded off anyway. You won’t always be so lucky, though. At some point in your future, you may need to display a number to the user. Imagine their look of shock as some crazy number with ten decimal places appears on the screen. “Oh my god! Why is it doing that?” exclaims a user of your cool new Flex app.
It gets worse. When these errors occur, checking if two numbers are equal can fail as well. Now, when you need to know if your function returned a multiple of two, and you get 15.9999999999, bad things can happen. Another problem, which I’ve run into a few times, is when I’m “counting” between two numbers, like two and three:
2 2.1 2.2 2.3 2.4 2.5000000001 2.6 2.7000000001 2.8 2.9 3
Consider the following two functions. I’ve been using them for a while now, and they work great. The first, correctFloatingPointError will manipulate the number (even a fractional one) with some clever multiplication, division, and rounding to try to account for the error. The second fuzzyEquals will check if two numbers are almost equal. In many cases, if two numbers have a difference of 0.00001, they might as well be considered equal. With both of these functions, you can manipulate the precision to allow for more accurate or inaccurate calculations depending on your needs.
/*
Copyright (c) 2007 Josh Tynjala
Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation
files (the "Software"), to deal in the Software without
restriction, including without limitation the rights to use,
copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.
*/
/**
* Corrects errors caused by floating point math.
*/
public function correctFloatingPointError(number:Number, precision:int = 5):Number
{
//default returns (10000 * number) / 10000
//should correct very small floating point errors
var correction:Number = Math.pow(10, precision);
return Math.round(correction * number) / correction;
}
/**
* Tests if two numbers are almost equal.
*/
public function fuzzyEquals(number1:Number, number2:Number, precision:int = 5):Boolean
{
var difference:Number = number1 - number2;
var range:Number = Math.pow(10, -precision);
//default check:
//0.00001 -0.00001
return difference -range;
}
If you have any more tips in this area, I’d like to hear them. Those two functions have been enough for my needs so far, but my toolbox always has room for an extra trick or two.
