Floating-Point and Decimal Arithmetic: Choose a Model

Floating-Point and Decimal Arithmetic

Floating-point and decimal arithmetic represent numbers according to different rules, and those rules affect equality, rounding, and accumulated error. Choose the representation from the invariant your application must preserve. Formatting a result to two places does not change the arithmetic that produced it.

Binary floating point is useful for many approximate numerical tasks. Decimal arithmetic can represent finite decimal inputs exactly, while still requiring a precision and rounding policy for operations that do not terminate or exceed the chosen context. Exact rational or integer representations solve other problems with different costs.

TL;DR

Define the required invariant before choosing a numeric type.
Use meaningful tolerances for approximate comparisons, including near zero.
Construct decimal values from intended decimal text when appropriate.
Specify precision, rounding, and the point where rounding occurs.
Validate finite values, magnitude, scale, and serialization boundaries.

Quick Example

These executed Python assertions expose representation and rounding differences. The tolerances and decimal policy are fictional teaching choices, not financial, legal, or scientific requirements for a real application.

Also test a negative halfway value and non-finite Decimal inputs. The results show this arithmetic in Python, not the numerical stability of an arbitrary algorithm or a suitable rounding policy for a regulated calculation.

Core Concepts

Python's floating-point tutorial explains that most decimal fractions do not have a finite binary representation. Stored values and displayed decimal text can therefore differ, and a short display can conceal the approximation. Python floating-point tutorial.

Python's Decimal documentation describes decimal representation, arithmetic contexts, precision, signals, and quantize for a chosen exponent. Constructing a Decimal from a float preserves that float's value rather than reconstructing the decimal text originally intended. Python Decimal documentation.

Distinguish representation error, measurement uncertainty, algorithmic instability, and deliberate rounding. Increasing precision can reduce some numerical error without making an uncertain measurement exact or repairing an unstable method.

Choose the invariant first

This is a selection aid. It does not prescribe a financial policy or claim one numeric type is universally faster or more accurate for every task.

For a fictional counter, an integer can preserve exact increments until its supported range is exceeded. For a sensor calculation, demanding exact equality between independently computed approximate values may be inappropriate. For a decimal input workflow, converting through binary float before Decimal can lose the intended representation at the entry point.

Compare approximate values with a problem-specific scale

Choose relative tolerance from acceptable proportional error and absolute tolerance from the smallest meaningful difference. Near zero, a purely relative test can fail to express the desired behavior because the reference magnitude is tiny. Units matter: a tolerance of one means something different for seconds, nanoseconds, and kilometers.

Approximate closeness is not a replacement for equality everywhere. It can be non-transitive: one value can be close to a second, and the second close to a third, while the first and third are not close. Avoid using a tolerance comparison as a drop-in equivalence relation for dictionary keys or grouping without a deliberate model.

Check non-finite values explicitly when the domain requires finite quantities. NaN and infinities can propagate through arithmetic and invalidate ordinary assumptions. Decide whether they are rejected, represented as missing, or supported with explicit semantics.

Make rounding and exchange part of the contract

Define when rounding happens. Rounding each line and then summing can differ from summing unrounded lines and rounding once. Neither sequence is automatically the correct business rule. Write the intended policy and verify it with domain reviewers where consequences require it.

Distinguish significant-digit precision from decimal places. A context precision limits significant digits in arithmetic; quantizing to a particular exponent addresses another requirement. A large magnitude can exhaust a context even when a value has few fractional places.

Choose an exchange format that preserves the intended information. If another component parses decimal text through binary float, the representation boundary may defeat your upstream choice. Test the complete write/read path with small values, large values, trailing zeros where meaningful, and halfway rounding cases.

Avoid treating a display string as the authoritative calculation state unless that is your explicit design. Formatting can hide error and remove information needed by later operations. Keep units and precision rules attached to the data model rather than scattered across screen rendering code.

Best Practices

Write a small set of invariant tests before implementing the calculation: exact totals where required, tolerances with units, defined rounding examples, and rejected non-finite or out-of-range inputs. Include negative values and boundary magnitudes rather than testing only friendly positive numbers.

Use stable numerical algorithms and appropriate libraries for the actual workload. A longer numeric type does not establish stability. Compare against an independent reference or analytical result where feasible, and distinguish finite test evidence from a general error bound.

Document type conversions at API, database, serialization, and presentation boundaries. A carefully selected numeric representation can be undermined by one implicit conversion far from the calculation itself.

Common Mistakes

Bad: Round the display and assume binary arithmetic became exact.

Correct: Choose the underlying model and calculation policy, then format the result for presentation.

Bad: Use one unexplained epsilon for every approximate comparison.

Correct: Derive tolerances from units, magnitude, and acceptable error, including behavior near zero.

Bad: Construct Decimal from a float when the requirement is exact decimal input.

Correct: Preserve the intended decimal text or exact source representation through the input boundary.

FAQ

Is Decimal always exact?

No. It can represent finite decimal inputs exactly, but arithmetic can still round under its context. Division such as one third has no finite decimal expansion.

Should approximate numbers ever use exact equality?

Sometimes, when the representation and operation contract justify it. Do not replace every comparison mechanically; identify which invariant the code needs.

What would reverse a decimal recommendation?

A workload dominated by approximate numerical operations, a required library interface, or unacceptable performance and memory costs may favor another representation. Preserve the actual accuracy requirement and measure the implementation under declared conditions.