Library/Numbers
From Athile
It may contain inaccurate, incorrect information. Use at your own risk.
Integers. Fixed point. Floating-point.
To understand floating-point numbers, it is easier to first consider how integers, real numbers, and fixed point numbers compare.
A real number, for the purposes here, is essentially the "standard" decimal number that most student learn in early stages of arithematic. A integer, by contrast, is a number that does not have a decimal component.
A floating-point number is an approximation of a real number that is can be implemented far more efficiently on a computer than a real number can be. Again, whereas integers and real numbers can both be thought of as pure mathematical concepts, a floating-point number is something that has arisen out of the practical needs of implementing a number system on a computer.
Contents |
Representing Real Numbers as Integers
Consider measuring a distance where only integers can be used. Let's say you measure a street length in meters. If the street length is actually 12.5 meters, but you are measuring in integers, you need either to truncate to 12 meters or round up to 13 meters. There's no way to express 12.5 as in integer.
One option would be to represent the distance as 25 meter divided by 2. That's fully accurate. But what happens if the real distance was 12.3379123? Without going into the detail, it should be fairly obvious why this representation (TODO: what is the formal name of this representation?) can quickly become cumbersome.
Another option is equivalent to fixed point math does: fixed point math says that it won't store all real numbers only a fixed number of decimal places. Therefore one integer could represent the full amount of meters and a second integer could represent the number of millimeters in the distance. This is effectively what fixed point math does, except that rather than storing all numbers as two integers, it "packs" both numbers into the same integer. To think of this "packing" differently (to show how trivial it is), what's the difference between storing a number as a meters distance plus a millimeters distance as opposed to just storing it in millimeters instead? There isn't any. Basically all fixed point math is saying is that it's in different units than a standard integer (it's units are a fraction of a standard int) and thus needs to be up or down scaled when interchanged with standard integers.
Floating Point
Since floating pointer has an imprecise representation where the rules of real number mathematics only approximately apply, a different coding strategy must be adopted.
Avoid Direct Equality Comparisons
- Test against an epsilon value instead
- The correct epsilon depends on every context - and is extremely difficult to determine
Normalize numbers before and after computations
Use unit'ed precision thresholds specified by the user
While an arbitrary operation on floating point numbers may be very difficult to determine, enforcing operations work within either application-wide or user-specified limits may easier in a sense since they are well-defined. For example, imagine the application guarantees that no single modelling operation will accumulate more than 1mm of error on each vertex. If this is known, an estimate based on the current model and the operation can likely meet this guarantee. If the threshold may be exceeded, then subdivision, renormalization, or alternate higher-precision mathematics can be used.
