Either people are doing very strange things with floating point numbers or this is generally pretty poor advice. You should not generally try to ham-fistedly pretend that approximate floating point operations are producing exact results. You should not generally be looking for exact equality of floating point values except in the case where you have a justified reason to believe that a floating point computation would arrive at an exact result. You should not generally be adding approximate equality terms to your directional comparisons because all that does is switch the precise point at which you are dividing the input space, except now it is at a location that isn't consistent with the opposite test and isn't specified explicitly in the program text. You also shouldn't treat exact floating point values as if they are approximate—floating point is not approximate, some floating point operations are (conditionally) approximate. A value of exactly 1.0, say from the result of min(1.0, f), is not going to arbitrarily change to a different value.
Decimal is also not a solution to floating point approximation. Decimal has all of the same rounding properties that binary floating point has, and actually it's a bit less well behaved. The differences are that Decimal round trips between textual decimal representations, and that Decimal supports an arbitrary choice of bit length.
There are cases where one does need to check for convergence to arbitrary values, but these are probably not what you are doing, and if they are you should probably know more about the precise numerical guarantees that you have.
Decimal representations have one really minor advantage: they behave more the way people expect with respect to which numbers are exactly representable especially in the size ranges of every-day numbers, since they exhibit mostly the same behavior as calculators do.
Plenty of people find that 0.2 not being exactly representable in binary floating point is not intuitive.
IEEE-754 allows "decimal floats". Basically, instead of 2^exponent, it's 10^exponent. `decimal32`[0], `decimal64`[1], and `decimal128`[2] are defined. But I'm not aware of any popular system that implements them.
> Decimal has all of the same rounding properties that binary floating point has
This is not true.
> Unlike hardware based binary floating point, the decimal module has a user alterable precision (defaulting to 28 places) which can be as large as needed for a given problem
> Both binary and decimal floating point are implemented in terms of published standards. While the built-in float type exposes only a modest portion of its capabilities, the decimal module exposes all required parts of the standard. When needed, the programmer has full control over rounding and signal handling. This includes an option to enforce exact arithmetic by using exceptions to block any inexact operations.
I think they mean in the abstract. With a fixed precision, you have round-off error as long as you only have a fixed number of digits, and thus have all the attendant issues in computations with them.
> Decimal is also not a solution to floating point approximation.
Then what is the solution?
As someone whose worked with finance where floating point approximation lead to one being attacked by auditor's chasing 1 cent error that somehow crept in over millions of operations, what other choices are there?
Floating point introduces the worst sort of error. If you are expecting exact results floating point gets it right with almost no effort 99.999% of the time. The remaining 0.001% it gets it wrong, and it happens at seemingly random. A heisenbug in other words. But if you force programmers to use integers, all those missing fractions of cents suddenly become glaringly obvious.
Decimals / fixed point / integers don't "solve" the approximation problem. They just make it plain, forcing programmers to think about it, code for it, and test for it. If you care about what happens to every last bit, then there is no other solution.
Or to put it another way, if you don't particularly care what happens to the lsb, then fine use floating point. Problems happen because people really do care, but are seduced by floating points 99.999% success rate and use it anyway. Hint: if you are comparing for the equality of two numbers computed in two different ways, you absolutely care.
I should have specified ‘decimal floating point’ for clarity, since that is what Python's Decimal is. I agree that non-floating representations often are the solution, depending on what problem you have.
>You should not generally be looking for exact equality of floating point values except in the case where you have a justified reason to believe that a floating point computation would arrive at an exact result.
I mean, yes, but there are plenty of cases where a result would be equal from an analytic perspective, but the implementation never realizes this due to floating point arithmetic.
>You also shouldn't treat exact floating point values as if they are approximate—floating point is not approximate, some floating point operations are (conditionally) approximate. A value of exactly 1.0, say from the result of min(1.0, f), is not going to arbitrarily change to a different value.
You're not wrong, but I think you also need to be careful here in how you communicate this. I've encountered plenty of situations where low-level details of floating point arithmetic mattered because it was only (in a sense) approximate, and I've even gotten non-deterministic behavior out of what should have been a deterministic program due to floating point issues.
I agree that you need to be cautious asserting that you know the exact value of a floating point value, because it isn't a property you get by default.
Generally, though, if it isn't fairly obvious that equality should be preserved, and if you don't have exactness as a requirement, then you probably shouldn't be building your solution around equality checks at all.
> You also shouldn't treat exact floating point values as if they are approximate—floating point is not approximate, some floating point operations are (conditionally) approximate.
Depending on your domain, they usually are. Perhaps your floating point number represents a reading from a sensor which has much less precision than your floating point type.
It would be more correct to model all of your approximate variables with error bars (and perhaps even error distributions) but in practice, treating the closest floating point number as a point estimate and doing all your comparisons to within some reasonable tolerance may be an acceptable approach.
This is a completely legitimate concern but it isn't about floating point. You could be getting integer readings from your sensor and you would have exactly the same concern. In such cases you need to provide tolerances in your code that correspond to the tolerances of your measurement. You might need to implement hysteresis, you might need to take multiple samples and track moving averages, you might need to take empirical measurements of the variance you expect and suppress readings below a threshold. It is very rare however for the solution to just be to say two values compare equal if they are within a billionth part from the other.
But why would you need to compare such measurements for equality with an arbitrary tolerance? If your algorithm is doing something like producing a yes/no answer from some measured data then it should be obvious what tolerance to use according to the domain. I would code that explicitly using inequalities instead of depending on some mystery default epsilon.
You’re right here I think, but it is a hard problem in practice I noticed. Hard in the engineering sense, because you’re sometimes writing generic library-level code that needs a domain specific notion of error bars and tolerances. So you need to parametrize everything, sometimes for multiple dimensions. Becomes messy quickly.
I must admit putting a mystery epsilon in my code here and there for that reason. Admittedly the wrong thing to do.
FWIW, I thought the advice in the article was generally correct (at least I couldn't spot anything I knew to be wrong). I agree that, if you actually need exactness and can spare the extra CPU cycles, floats are not your best bet (e.g. banking, accounting). But for the use case:
> There are cases where one does need to check for convergence to arbitrary values [...]
the advice in the article doesn't strike me as wrong per se. If you're implementing a numerical algorithm for some reason and you want to check that it's well behaved, adding a couple of tests where you check for approximate equality of some results as detailed in the article seems fine to me. (Of course, ideally you would additionally also formally prove the numerical stability of that algorithm.)
I guess the article is maybe missing one crucial piece of advice, namely that floating point computations are not guaranteed to be well-behaved. If you don't know much about floating point computations and just read this article, you might be tempted to think that every computation involving floats should lead to a result that is "close" to the "real" result. But that's not true, numerical algorithms can be poorly behaved and error terms can multiply, leading to catastrophically wrong results.
> I agree that, if you actually need exactness and can spare the extra CPU cycles, floats are not your best bet (e.g. banking, accounting).
You are misunderstanding me, this is not what I was saying at all. I think much of the time when people feel like they should use floats they can actually use floats, yet only in very rare, niche scenarios should math.isclose be used.
Using exact equality comparisons with floating point numbers, while fairly uncommon, seems a lot more frequently the right thing to do than math.isclose: for example, sorting algorithms, hashmaps and caches, checking for equality between structs, sentinel values.
Even in most cases where approximate equality is wanted, like maybe you are snapping a position to another position, or maybe you are checking that your new audio processing algorithm is producing noise below a noise floor, even then I would hesitate to use math.isclose, because it still probably isn't a good fit for the domain, particularly with it having an awkward default relative error term.
> Using exact equality comparisons with floating point numbers, while fairly uncommon, seems a lot more frequently the right thing to do than math.isclose: for example, sorting algorithms, hashmaps and caches, checking for equality between structs, sentinel values.
I mean, math.isclose() doesn't even form an equivalence relation, so of course it doesn't make sense to base struct or hash value equality off of it.
I don't think anyone was suggesting to use approximate equality as a form of actual equality. The only reasonable equality for floats is exact equality, yes. However, it's also an equality which is useless for calculation.
> I think much of the time when people feel like they should use floats they can actually use floats
I disagree, many LOB applications deal almost exclusively with discrete settings, and in such cases, floats are generally not necessarily appropriate. Money, for example, should generally not be represented as a float, unless you're doing e.g. financial mathematics on it, estimating return rates and so on (and then with the understanding that these calculations won't be exact).
I think most people trying to losslessly represent money or similar discrete quantities already have the intuition that you shouldn't use binary floating point, and those that don't are not well served by telling them to math.isclose it, which is at best a fragile bandaid.
What a strange comment to say "A value of exactly 1.0, say from the result of min(1.0,f), is not going to arbitrarily change to a different value." Christ man, floating point arithmetic is on computers, it's definitely deterministic, no one is saying it's indeteriminable, lol, including the author. It's not like we're off in LSD space where numbers change meaning from moment to moment, it's only that there is information lost in each operation that lead to accumulating error such that exact comparisons usually do not express what you mean to compare in computer programs. That's all, and I don't think the OP is saying anything else.
Other than that I'm not sure what you're adding. The point is people are unclear when they write code because they do not understand floats, and because few people really understand round-off error, they should as a rule compare with np.isclose unless, as you say, they now for a fact they expect a specific floating point number (although the times that happens is by and far the minority in actual code).
Alas, floating point arithmetic is not necessarily deterministic in the sense that running the same Python or C code twice is guaranteed to give you the some results.
First, your compiler is often allowed to do weird things. (Like compile the same code in different ways.) And there's some weirdness with the state of flags that might be different from iteration to iteration.
Yeah, pretty much the only case I can think of when you'd want to check floating points for equality is for bug hunting e.g. if you calculate a set of disjoint probabilities you might want to verify that they add up to 1.
Eg when you want to add a list of (non-negative) numbers, it makes a difference what order you add them in. The usual recommendation is to add them in ascending order.
It's not commutative, yes, but it should still obey the property that the relative error terms add instead of multiplying, no?
(i.e. in the parent's case of "checking that, if you add a bunch of disjoint probabilities, they're at most 1", doing that with approximate equality seems fine to me?)
Decimal is also not a solution to floating point approximation. Decimal has all of the same rounding properties that binary floating point has, and actually it's a bit less well behaved. The differences are that Decimal round trips between textual decimal representations, and that Decimal supports an arbitrary choice of bit length.
There are cases where one does need to check for convergence to arbitrary values, but these are probably not what you are doing, and if they are you should probably know more about the precise numerical guarantees that you have.