Floating Point Precision Calculator
Determine the precision and representable range of a floating point format.
About Floating Point Precision Calculator
Floating point precision is relative, so the absolute gap between representable values grows with magnitude. Errors accumulate roughly as the square root of the operation count under random rounding. Money should never be held in binary floating point, because values like 0.1 have no exact representation and errors compound across arithmetic.
Formula
Floating Point Precision
Epsilon = 2 to the power of minus mantissa bits; digits = mantissa bits × log₁₀2
- machine epsilon
- The smallest relative gap between representable values
- representation gap
- Absolute spacing between neighbours, which grows with magnitude
Worked example
With the default values below, Floating Point Precision Calculator returns Significant Decimal Digits of 15.65. Change any input above and every figure here is recalculated in your browser.
Inputs used
- Format width in bitsaccepts 16–128 bits
- 64 bits
- Magnitude of the value storedminimum 0
- 1,000,000
- Number of chained operationsminimum 1
- 1,000
Results
- Significant Decimal Digits
- 15.65
- Machine Epsilon
- 0.00000000000000022204
- Representation Gap at That Magnitude
- 0.000000000116
- Estimated Accumulated Error
- 0.000000003681
- Guidance
- Double precision gives about sixteen digits, adequate for most scientific work
Step by step
- 1Values used
- 2Floating Point Precision
- 3Significant Decimal Digits= 15.65
- 4Machine Epsilon= 0.00000000000000022204
- 5Representation Gap at That Magnitude= 0.000000000116
- 6Estimated Accumulated Error= 0.000000003681
- 7Guidance= Double precision gives about sixteen digits, adequate for most scientific work
Assumptions and limits
- The result is derived only from the 3 inputs listed above. Anything not among them is not part of the calculation.
- In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
- Figures are rounded for display to 20 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.
How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.
Frequently Asked Questions
How is Floating Point Precision calculated?
Epsilon = 2 to the power of minus mantissa bits; digits = mantissa bits × log₁₀2 Floating point precision is relative, so the absolute gap between representable values grows with magnitude. Errors accumulate roughly as the square root of the operation count under random rounding.
Why does Floating Point Precision matter?
Money should never be held in binary floating point, because values like 0.1 have no exact representation and errors compound across arithmetic.
What values do I need to enter?
This calculator takes 3 inputs: Format width in bits, Magnitude of the value stored, Number of chained operations. The pre-filled defaults are a realistic starting point — replace them with figures from your own environment for a result you can act on.